Scanning system, scanned image processing device and scanning method

ABSTRACT

A scanning system according to the present invention includes an image acquisition unit that acquires a plurality of pieces of image information generated by continuously scanning a medium to be scanned, a path calculation unit that calculates a path of scanning the medium based on the plurality of pieces of image information acquired by the image acquisition unit, a processing method determination unit that determines a processing method of the plurality of pieces of image information in accordance with a path calculated by the path calculation unit, and a processing unit that processes the plurality of pieces of image information by a processing method determined by the processing method determination unit and converts the plurality of pieces of image information into information in a form corresponding to the processing method.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2015-242277, filed on Dec. 11, 2015, thedisclosure of which is incorporated herein in its entirety by reference.

BACKGROUND

The present invention relates to a scanning system, a scanned imageprocessing device and a scanned method and, for example, relates to atechnique to scan a medium.

There are a considerable number of people who, during reading, addmarkings with a pen to information contained in a paper book, such as apart that they think is important or have a question about in a text(body), or a drawing.

On the other hand, a marking (highlight) function is provided inelectronic books. In electronic books, because the content of a book hasbeen converted into electronic data, the text extracted by a user can beeasily stored and searched with use of the highlight function.

For paper books also, there is a need to easily convert a part of apaper book into electronic form and store it, like electronic books. Torespond to such a need, Japanese Unexamined Patent Publication No.2004-30325 discloses a technique that aims at enabling the conversion ofdata between a paper document and an electronic document with use of anuncomplicated system.

A calendar book system disclosed in Japanese Unexamined PatentPublication No. 2004-303254 is composed of a pen device and a base unit.The pen device includes three optical sensors (CCD arrays). The firstoptical sensor is mounted on a linear side along the length of the pendevice. The second optical sensor is mounted on the pen point of the pendevice or one side of the pen point. The third optical sensor is mountedon the rear end of the pen device. The first optical sensor is used toscan a large part of a page, for example. The second optical sensor isused to identify the local content of a document by recognizing aprinted mark, for example.

The output of the optical sensors is sent as a sensor signal to the baseunit by a transmitter of the pen device. The base unit communicates withthe pen device and receives the sensor signal. The base unit can therebycapture the user's document as an electronic document.

SUMMARY

A paper book contains different types of information, such as a body, adrawing and a page number of a page on which the body or the drawing ispresented. There is a need to store each of the different types ofinformation in an appropriate form according to the type. For example,some users wish to store a body and a page number in the form of anelectronic text and store a drawing in the form of an electronic image.

However, according to the technique disclosed in Japanese UnexaminedPatent Publication No. 2004-30325, a scanned text can be stored only inthe form of an electronic document. In other words, Japanese UnexaminedPatent Publication No. 2004-30325 does not disclose a technique to storedifferent types of information on a medium to be scanned in appropriateforms.

The other problems and novel features of the present invention willbecome apparent from the description of the specification and theaccompanying drawings.

According to one embodiment, a scanning system processes a plurality ofpieces of image information generated by continuously scanning a mediumto be scanned by a processing method determined in accordance with apath of scanning the medium, and thereby converts the plurality ofpieces of image information into information in a form corresponding tothe processing method.

According to the above embodiment, it is possible to store differenttypes of information on a medium to be scanned in appropriate forms witha simple operation.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, advantages and features will be moreapparent from the following description of certain embodiments taken inconjunction with the accompanying drawings, in which:

FIG. 1 is a view showing a schematic configuration of a scanning systemaccording to a first embodiment.

FIG. 2 is a view showing a hardware configuration of a pen scanneraccording to the first embodiment.

FIG. 3 is a view showing the pen-up state and the pen-down state of thepen scanner according to the first embodiment.

FIG. 4 is a state transition diagram of the pen scanner according to thefirst embodiment.

FIG. 5 is a view showing a calculation method of a moving path of thepen scanner and a vector according to the first embodiment.

FIG. 6 is a view schematically showing a calculation method of a movingpath of the pen scanner according to the first embodiment.

FIG. 7 is a view showing a detailed configuration of the scanning systemaccording to the first embodiment.

FIG. 8 is a view showing a calculation method of a vector of a movingpath, and a determination method of the moving path, of the pen scanneraccording to the first embodiment and a determination method of themoving path.

FIG. 9 is a flowchart showing a scanning process of the scanning systemaccording to the first embodiment.

FIG. 10 is a view to describe effects of the scanning system accordingto the first embodiment.

FIG. 11 is a view showing a hardware configuration of a pen scanneraccording to a second embodiment.

FIG. 12 is a view showing the relationship between an operating stateand a moving path and a processing method according to the secondembodiment.

FIG. 13 is a view showing a detailed configuration of the scanningsystem according to the second embodiment.

FIG. 14 is a flowchart showing a scanning process of the scanning systemaccording to the second embodiment.

FIG. 15 is a view showing a detailed configuration of a scanning systemaccording to a third embodiment.

FIG. 16 is a flowchart showing a scanning process of the scanning systemaccording to the third embodiment.

FIG. 17 is a view showing one example of information to be stored in astorage means of a server according to the third embodiment.

FIG. 18 is a view showing a detailed configuration of a scanning systemaccording to a fourth embodiment.

FIG. 19 is a flowchart showing a scanning process of the scanning systemaccording to the fourth embodiment.

FIG. 20 is a view showing a detailed configuration of a scanning systemaccording to a fifth embodiment.

FIG. 21 is a flowchart showing preprocessing of the scanning systemaccording to the fifth embodiment.

FIG. 22 is a flowchart showing a scanning process of the scanning systemaccording to the fifth embodiment.

FIG. 23 is a view showing one example of information to be stored in astorage means of a server according to the fifth embodiment.

FIG. 24 is a view showing a detailed configuration of a scanning systemaccording to a first modified example of the fifth embodiment.

FIG. 25 is a flowchart showing a scanning process in a scan modeaccording to the first modified example of the fifth embodiment.

FIG. 26 is a view showing a detailed configuration of a server accordingto the first modified example of the fifth embodiment.

FIG. 27 is a view showing a hardware configuration of a pen scanneraccording to a sixth embodiment.

FIG. 28 is a view showing a detailed configuration of a scanning systemaccording to the sixth embodiment.

FIG. 29 is a view showing a detailed configuration of a scanning systemaccording to a seventh embodiment.

FIG. 30 is a view showing the relationship between information that isinput by scanning and information that is output according to thatinformation in a viewing mode according to the seventh embodiment.

FIG. 31 is a view to describe effects of the scanning system accordingto the seventh embodiment.

FIG. 32 is a view showing a processing method of a scanning systemaccording to an eighth embodiment.

FIG. 33 is a view showing a detailed configuration of the scanningsystem according to the eighth embodiment.

FIG. 34 is a flowchart showing a scanning process in a scan modeaccording to a first modified example of the eighth embodiment.

FIG. 35 is a flowchart showing a scanning process in a scan modeaccording to the first modified example of the eighth embodiment.

FIG. 36 is a view showing a configuration of a scanning system, which isa schematic configuration according to an embodiment.

DETAILED DESCRIPTION

Preferred embodiments of the present invention will be describedhereinafter with reference to the drawings. It should be noted thatspecific numerical values and the like in the following embodiments aregiven merely for illustrative purposes, and values are not limitedthereto unless particularly noted. Further, in the following descriptionand drawings, things that are obvious to those skilled in the art andthe like are appropriately omitted, shortened and simplified to clarifythe explanation.

First Embodiment

The schematic configuration of a scanning system 1 according to a firstembodiment is described hereinafter with reference to FIG. 1. As shownin FIG. 1, the scanning system 1 includes a pen scanner 2, a smartphone3, and a server 4.

The pen scanner 2 and the smartphone 3 transmit and receive arbitraryinformation to and from each other via arbitrary wireless communication,for example. As the wireless communication, near field radiocommunication such as Bluetooth or LAN (Local Area Network) can be used.Further, the smartphone 3 and the server 4 transmit and receivearbitrary information to and from each other via arbitrary wirelesscommunication and arbitrary wired communication, for example. As thewireless communication, mobile communication (long range radiocommunication) such as 3GPP (Third Generation Partnership Project) orLTE (Long Term Evolution), for example, can be used. As the wiredcommunication, Internet communication, for example, can be used.

The pen scanner 2 is a device that scans a medium to be scanned (whichis also referred to simply as “medium”). The pen scanner 2 scans anarbitrary part of a medium in response to a user's operation andgenerates image information where the scanned part is converted into anelectronic image. When a user slides the pen scanner 2 over a part of amedium to be scanned, the pen scanner 2 continuously scans this part andgenerates a plurality of pieces of image information. The pen scanner 2transmits the plurality of pieces of generated image information to thesmartphone 3.

At this time, the pen scanner 2 calculates the moving path of the penscanner 2 when scanning the medium based on the plurality of pieces ofgenerated image information. The pen scanner 2 transmits informationabout the calculated moving path also to the smartphone 3.

The medium to be scanned is typically a paper book. When a user wishesto store information in electronic form into the server 4, the userscans an arbitrary part (a body, a page number, a drawing etc.) of apaper book with the pen scanner 2.

The smartphone 3 is an information processing device that performsspecified processing on a plurality of pieces of image informationreceived from the pen scanner 2. The smartphone 3 determines aprocessing method to be performed on the plurality of pieces of imageinformation from a plurality of processing methods that are prepared inadvance, based on the information about the moving path received fromthe pen scanner 2. Then, the smartphone 3 processes the plurality ofpieces of image information by the determined processing method andthereby converts the information into a form corresponding to theprocessing method.

As specified processing for converting information, OCR (OpticalCharacter Recognition), conversion into JPG (Joint Photographic expertsGroup) and the like are prepared. Specifically, as a result that a userscans different types of information by different moving paths, thedifferent types of information can be converted into different forms ofinformation. It is thereby possible to store different types ofinformation in appropriate forms.

The smartphone 3 adds an attribute to the converted information andtransmits them to the server 4. Specifically, the smartphone 3 adds, tothe converted information, attribute information indicating the type (abody, a drawing, a page number etc.) of the information and transmitsthem to the server 4.

Although an example of using a smartphone as an information processingdevice that processes image information generated by the pen scanner 2and transmits it to the server 4 is described hereinafter in thisembodiment, the type of the information processing device is not limitedthereto. As the information processing device, a PC (Personal Computer),an information terminal with a touch panel which is generally called“tablet” or the like may be used, for example. When the informationprocessing device is a PC, the PC and the server 4 are configured to beable to communicate with each other via wired communication (forexample, Internet communication). On the other hand, when theinformation processing device is a tablet, the tablet and the server 4are configured to be able to communicate with each other via wirelesscommunication (for example, wireless LAN) and wired communication (forexample, Internet).

The server 4 is an information processing device that stores theconverted information received from the smartphone 3. The server 4 cancollectively store a plurality of pieces of converted informationrespectively obtained as a result of scanning by a plurality of users.

For example, the pen scanner 2 and the smartphone 3 described above areowned by the same user. Another user can also own the pen scanner 2 andthe smartphone 3 described above. Thus, the server 4 can store aplurality of pieces of converted information respectively received froma plurality of smartphones 3 owned by different users.

The hardware configuration of the pen scanner 2 according to the firstembodiment is described hereinafter with reference to FIG. 2. As shownin FIG. 2, the pen scanner 2 includes a lens 10, an image sensor 11, apen-down detection device 12, an MCU 13, a memory 14, and a transceiver15.

The lens 10 forms an image of a part of a medium that is illuminated bya light source (not shown) on the image sensor 11. The image sensor 11photoelectrically converts the image formed by the lens and therebygenerates image information. Stated differently, the image sensor 11takes an image of (scans) a part of a medium (for example, a part of apage of a paper book), generates image information presenting the partof the medium as an electronic image, and outputs it to the MCU 13. Oneimage presented by one piece of image information is an image showing aselected character of a text (for example, one character or a part ofone character) written in a paper book or a part of a drawing, forexample. The image sensor 11 generates the image information byperforming scanning at a specified time interval. Hereinafter, eachimage at a specified time interval is also referred to as “frame”.

The pen-down detection device 12 is a device that detects that the penscanner 2 is in the pen-down state. Stated differently, the pen-downdetection device 12 determines whether an imaging unit (the lens 10 andthe image sensor 11) mounted on the pen point of the pen scanner 2 is inclose proximity to a medium at a predetermined distance or less. As thepen-down detection device 12, a switch or a pressure sensor, forexample, may be used.

(1) When a Switch is Used as the Pen-Down Detection Device 12

A switch generates a notification signal indicating whether the switchis pressed or not and outputs it to the MCU 13. When a contact surfaceof the switch is pressed against a medium, that is, when the switch ispressed, the switch generates a notification signal indicating that theswitch is pressed and outputs it to the MCU 13. On the other hand, whencontact surface of the switch is not pressed against a medium, that is,when the switch is not pressed, the switch generates a notificationsignal indicating that the switch is not pressed and outputs it to theMCU 13.

When the notification signal indicating that the switch is pressed isinput from the switch, the MCU 13 determines that the pen scanner 2 isin the pen-down state. On the other hand, when the notification signalindicating that the switch is not pressed is input from the switch, theMCU 13 determines that the pen scanner 2 is in the pen-up state.

(2) When a Pressure Sensor is Used as the Pen-Down Detection Device 12

A pressure sensor detects a pressure that is applied from a medium,generates a notification signal indicating the detected pressure andoutputs it to the MCU 13. When the pressure indicated by thenotification signal input from the pressure sensor is equal to or morethan a specified threshold, the MCU 13 determines that the pen scanner 2is in the pen-down state. On the other hand, when the pressure indicatedby the notification signal input from the pressure sensor is less than aspecified threshold, the MCU 13 determines that the pen scanner 2 is inthe pen-up state.

The MCU 13 is a device that controls the pen scanner 2. For example, theMCU 13 acquires the image information generated by the image sensor 11and stores it into the memory 14. The memory 14 is a storage device inwhich various types of data are stored.

The transceiver 15 is a device that wirelessly transmits and receivesvarious types of data to and from the smartphone 3. For example, thetransceiver 15 converts the image information stored in the memory 14from an electrical signal to a radio signal and transmits it to thesmartphone 3.

In this configuration, when a user brings the pen point of the penscanner 2 close to a medium in order to scan the medium as shown on theright of FIG. 3, the pen-down detection device 12 is pushed against themedium. The MCU 13 can thereby recognize the pen-down state of the penscanner 2. On the other hand, when a user brings the pen point of thepen scanner 2 away from a medium in order to stop scanning the medium asshown on the left of FIG. 3, the pen-down detection device 12 is notpushed against the medium. The MCU 13 can thereby recognize the pen-upstate of the pen scanner 2.

The state transition of the pen scanner 2 according to the firstembodiment is described hereinafter with reference to FIG. 4. The penscanner 2 transitions between four states: “idle state”, “initializationstate”, “image and path acquisition state” and “analysis executionstate”. The following description is based on the assumption that theinitial state of the pen scanner 2 is “idle state”.

In “idle state”, the MCU 13 of the pen scanner 2 does not store theimage information generated by the image sensor 11 into the memory 14.When the MCU 13 determines that the pen scanner 2 is put into thepen-down state based on the notification signal from the pen-downdetection device 12, the MCU 13 transitions to “initialization state”(S1).

In “initialization state”, the MCU 13 initializes the memory 14. To bespecific, the MCU 13 deletes, from the memory 14, image information,path information and vector information that have been stored in thememory 14 in the previous scanning. Note that the path information andthe vector information are described later. After initializing thememory 14, the MCU 13 transitions to “image and path acquisition state”(S2).

In “image and path acquisition state”, the MCU 13 stores the imageinformation generated by the image sensor 11 into the memory 14.Specifically, as described above, the image sensor 11 generates theimage information at a specified time interval. Each time the imageinformation is generated, the MCU 13 stores the image information one byone into the memory 14. When the MCU 13 determines that the pen scanner2 is put into the pen-up state based on the notification signal from thepen-down detection device 12, the MCU 13 transitions to “analysisexecution state” (S3).

When the MCU 13 transitions to “analysis execution state”, the MCU 13stops storing the image information generated by the image sensor 11into the memory 14. Thus, a plurality of pieces of image informationthat have been generated by the image sensor 11 during the period frompen-down to pen-up of the pen scanner 2 are stored in the memory 14.

In “analysis execution state”, the MCU 13 analyzes the plurality ofpieces of image information stored in the memory 14 during “image andpath acquisition state”, and calculates the moving path of the penscanner 2 and its vector. To be specific, the analysis is executed inthe procedure shown in FIG. 5.

(1) First, the MCU 13 extracts a feature point in each of a plurality ofimages (a plurality of frames) presented by a plurality of pieces ofimage information. (2) Next, the MCU 13 calculates the moving path ofthe pen scanner 2 by checking the same feature point among a pluralityof frames. For example, the MCU 13 calculates, as the moving path of thepen scanner 2, a line that connects the points at the same coordinateposition in the respective frames (which are the black points at theupper left of the respective frames in the example of FIG. 5) when aplurality of frames are superimposed on one another with the positionsof the feature point that is the same among the plurality of framesbeing aligned. The MCU 13 generates path information indicating thecalculated moving path and stores the path information into the memory14 in association with the plurality of pieces of image information usedfor the calculation of the path information. (3) Then, the MCU 13applies linear approximation to the moving path of the pen scanner 2 andthereby calculates a vector (so-called motion vector) indicating themotion of the pen scanner 2. The MCU 13 generates vector informationindicating the calculated vector and stores the vector information intothe memory 14 in association with the plurality of pieces of imageinformation used for the calculation of the vector information.

The calculation of the vector is performed for every specified number offrames. Stated differently, for a plurality of pieces of imageinformation stored in the memory 14, the MCU 13 calculates a vector fromthe points calculated as the moving path for every specified number ofpieces of image information. A plurality of pieces of image informationstored in the memory 14 may be sorted into groups of a specified numberon the time series, and the moving path and the vector may be calculatedfor each group. Further, a specified number of pieces of imageinformation that form a group for the calculation of the moving path andthe vector may partly overlap with a specified number of pieces of imageinformation that form another group for the calculation of the movingpath and the vector. Note that, although FIG. 5 illustrates an examplewhere the specified number is 5 (for example, time t=0 to time t=4 inFIG. 5), it is not limited thereto.

A calculation method of the moving path of the pen scanner 2 isspecifically described hereinafter with reference to FIG. 6. As a methodof calculating the moving path of the pen scanner 2, a method that isused in an optical mouse may be employed.

As shown in FIG. 6, a frame contains a plurality of pixels. AlthoughFIG. 6 shows a frame of 6×6 pixels for simplification of thedescription, an actual frame size is larger than this size as a matterof course. Further, it is assumed in FIG. 6 that the axis that isparallel to the vertical direction is the Y axis, and the axis that isparallel to the horizontal direction is the X axis. As shown in FIG. 6,the MCU 13 extracts, as a feature point, a point that is distinguishablefrom the other part of a medium by a difference in density. Thedifference in density is caused by the unevenness on a medium, thepresence or absence of a character and the like, for example.

In FIG. 6, from time t=0 to time t=1, the feature point moves by +2 inthe X direction and +1 in the Y direction within the frame. From timet=1 to time t=2, the feature point moves by +2 in the X direction and −1in the Y direction within the frame. In this case, the moving path ofthe same coordinate position in the respective frames (which is theuppermost and rightmost pixel in the example of FIG. 6) when a pluralityof frames are superimposed on one another with the positions of thefeature point that is the same among the plurality of frames beingaligned is obtained as the moving path of the pen scanner 2.Specifically, the moving path that moves by −2 in the X direction and −1in the Y direction from time t=0 to time t=1 and moves by −2 in the Xdirection and +1 in the Y direction from time t=1 to time t=2 isobtained as the moving path of the pen scanner 2.

When the calculation of the vector ends, the MCU 13 transmits aplurality of pieces of image information and a plurality of pieces ofvector information stored in the memory 14 to the smartphone 3 throughthe transceiver 15. The smartphone 3 thereby determines a processingmethod to be performed on the plurality of pieces of image informationbased on the plurality of pieces of vector information received from thepen scanner 2.

Note that, although an example where the path information is generatedby the MCU 13 is described above, it is not limited thereto. Forexample, an image sensor of an optical mouse may be used as the imagesensor 11, and the path information that is generated by a DSP (DigitalSignal Processor) included in the image sensor may be acquired.

The detailed configuration of the scanning system 1 according to thefirst embodiment is described hereinafter with reference to FIG. 7.

As shown in FIG. 7, the pen scanner 2 includes an image acquisitionmeans 20, a path calculation means 21, and a vector calculation means22.

The image acquisition means 20 scans a medium during the period when thepen scanner 2 is in the pen-down state, and thereby sequentiallygenerates image information representing the medium as an image andholds them. Specifically, the image acquisition means 20 generates aplurality of pieces of image information presenting a part of a mediumthat has been scanned when the pen scanner 2 is in the pen-down state asa plurality of images and holds them. When the pen scanner 2 is put intothe pen-up state, the image acquisition means 20 transmits the pluralityof pieces of held image information to the smartphone 3. Specifically,the lens 10, the image sensor 11, the pen-down detection device 12, theMCU 13, the memory 14 and the transceiver 15 operate as the imageacquisition means 20.

The path calculation means 21 calculates the moving path of the penscanner 2 based on the plurality of pieces of image information held bythe image acquisition means 20 when the pen scanner 2 is put into thepen-up state. Specifically, the pen-down detection device 12, the MCU 13and the memory 14 operate as the path calculation means 21.

The vector calculation means 22 calculates a plurality of vectors in themoving path of the pen scanner 2 based on the moving path of the penscanner 2 calculated by the path calculation means 21, and generates aplurality of pieces of vector information indicating the plurality ofcalculated vectors. The vector calculation means 22 transmits theplurality of pieces of generated vector information to the smartphone 3.Specifically, the MCU 13, the memory 14 and the transceiver 15 operateas the vector calculation means 22.

Further, as shown in FIG. 7, the smartphone 3 includes a processingmethod determination means 30, a first selection means 31, a firstprocessing means 32, a second processing means 33, and a secondselection means 34.

The processing method determination means 30 receives the plurality ofpieces of vector information transmitted from the vector calculationmeans 22 of the pen scanner 2. The processing method determination means30 determines whether the moving path of the pen scanner 2 is linear ornon-linear based on the plurality of vectors indicated by the pluralityof pieces of received vector information.

When the processing method determination means 30 determines that themoving path of the pen scanner 2 is linear, it determines to process theplurality of pieces of image information received from the pen scanner 2by a first processing method performed by the first processing means 32.On the other hand, when the processing method determination means 30determines that the moving path of the pen scanner 2 is non-linear, itdetermines to process the plurality of pieces of image informationreceived from the pen scanner 2 by a second processing method performedby the second processing means 33.

A method of calculating the vectors of the moving path of the penscanner 2 and a method of determining whether the moving path of the penscanner 2 is linear or non-linear are specifically described hereinafterwith reference to FIG. 8.

For example, as shown in FIG. 8, it is assumed that a user continuouslyscans a medium during the period from pen-down to pen-up. As describedearlier, a plurality of vectors of the moving path of the pen scanner 2are calculated for every specified number of frames. The processingmethod determination means 30 calculates the inner product of thevectors calculated in this manner, for example, and thereby determineswhether the moving path of the pen scanner 2 is linear or non-linear.

The processing method determination means 30 selects two vectors fromthe plurality of vectors and calculates the inner product of the twoselected vectors. In a simple way, when the inner product of the twovectors is equal to the product of the size of those two vectors, it canbe determined that those two vectors are linear, and otherwise it can bedetermined that those two vectors are non-linear. However, in the casewhere vectors are determined to be non-linear by such a determinationmethod, even when a user intends to move the pen scanner 2 linearly,they are determined to be non-linear if the moving path is curved onlyslightly. Therefore, it is preferred to determine that two vectors arelinear when the proportion of the inner product of the two vectors tothe product of the size of those two vectors is equal to or more than aspecified value. On the other hand, it is preferred to determine thattwo vectors are non-linear when the proportion of the inner product ofthe two vectors to the product of the size of those two vectors is lessthan a specified value. For example, in the case where an error withwhich a vector is regarded as being linear when a user scans a mediumwith the pen scanner 2 is 15°, the above proportion may be 0.966 (=cos15°).

Stated differently, when an angular difference between two vectors isequal to or less than a specified angle, the processing methoddetermination means 30 determines that those two vectors are linear. Onthe other hand, when an angular difference between two vectors is morethan a specified angle, the processing method determination means 30determines that those two vectors are non-linear.

As a pair of two vectors to calculate the inner product, any pair may beemployed. For example, the inner product may be calculated for allpossible pairs formed by selecting two vectors from a plurality ofvectors, or the inner product may be calculated for pairs formed byselecting two vectors that are temporally successive.

Then, when an angular difference between two vectors is equal to or lessthan a specified angle for all pairs, the processing methoddetermination means 30 determines that the moving path of the penscanner 2 is linear. On the other hand, when an angular differencebetween two vectors is more than a specified angle for at least onepair, the processing method determination means 30 determines that themoving path of the pen scanner 2 is non-linear.

The first selection means 31 receives the plurality of pieces of imageinformation transmitted from the image acquisition means 20 of the penscanner 2. The first selection means 31 outputs the plurality of piecesof received image information to one of the first processing means 32and the second processing means 33 in accordance with a determinationresult of the processing method determination means 30. To be specific,when the processing method determination means 30 determines to processthe plurality of pieces of image information by a first processingmethod, the first selection means 31 outputs the plurality of pieces ofreceived image information to the first processing means 32. On theother hand, when the processing method determination means 30 determinesto process the plurality of pieces of image information by a secondprocessing method, the first selection means 31 outputs the plurality ofpieces of received image information to the second processing means 33.

As the first processing method, the first processing means 32 combines aplurality of images presented by the plurality of pieces of imageinformation, performs OCR on a character sequence contained in the imagegenerated by the combining, generates text information where thecharacter sequence is presented in the form of an electronic text, andadds an attribute to the generated text information. For example, thetext information is a text file (with the extension “txt”, for example),a file that can be used with an arbitrary text editor (with theextension “doc” or “docx”, for example) or the like.

As the second processing method, the second processing means 33 combinesa plurality of images presented by the plurality of pieces of imageinformation, compresses the image generated by the combining, generatescompressed image information presenting the compressed image, and addsan attribute to the generated compressed image information. For example,the generated compressed image information is a JPG file, a GIF(Graphics Interchange Format) file, a PNG (Portable Network Graphics)file or the like. Hereinafter, an example that generates a JPG file asthe compressed image information is described in this embodiment.

Each of the first processing means 32 and the second processing means 33combines a plurality of images in the state where those images aresuperimposed on one another with the positions of the feature point thatis the same among the plurality of images being aligned. In this manner,it is possible to combine a plurality of images by superimposing a partthat is the same among the plurality of images on one another.

The second selection means 34 acquires the text information or thecompressed image information that is generated by one of the firstprocessing method performed by the first processing means 32 and thesecond processing method performed by the second processing means 33 inaccordance with a determination result of the processing methoddetermination means 30. The second selection means 34 transmits theacquired text information or compressed image information to the server4.

A CPU (Central Processing Unit) included in the smartphone 3, forexample, executes a program stored in a storage means (not shown)included in the smartphone 3 and thereby operates as the above-describedmeans 30 to 34. In other words, this program contains a plurality ofinstructions to cause the CPU to perform the processing as theabove-described means 30 to 34. Further, the storage means includes atleast one of storage devices such as a volatile memory, a hard disk, anda flash memory (nonvolatile memory), for example.

As shown in FIG. 7, the server 4 includes an information control means40 and a storage means 41.

The information control means 40 receives the text information or thecompressed image information transmitted from the second selection means34 of the smartphone 3. The information control means 40 stores thereceived text information or compressed image information into thestorage means 41.

The storage means 41 stores the text information or the compressed imageinformation as described above. The storage means 41 includes at leastone of storage devices such as a volatile memory, a hard disk, and aflash memory (nonvolatile memory), for example.

A CPU (Central Processing Unit) included in the server 4, for example,executes a program stored in the storage means 41 included in the server4 and thereby operates as the above-described information control means40. In other words, this program contains a plurality of instructions tocause the CPU to perform the processing as the above-describedinformation control means 40.

A scanning process of the scanning system 1 according to the firstembodiment is described hereinafter with reference to FIG. 9. In thefollowing description, it is assumed that, when a user scans a pagenumber of a paper book, the user scans the page number by moving the penscanner 2 in a linear moving path. It is also assumed that, when a userscans a body of a paper book, the user scans the body by moving the penscanner 2 in a non-linear moving path.

The image acquisition means 20 of the pen scanner 2 waits for a user toperform an operation of scanning a medium with the pen scanner 2 (S11and No in S12). Specifically, the image acquisition means 20 waits forthe pen scanner 2 to enter the pen-down state.

When the user performs an operation to scan a medium with the penscanner 2 (Yes in S12), the image acquisition means 20 starts thescanning of the medium (S13). The image acquisition means 20 repeats thegeneration and holding of the image information presenting the scannedimages of the medium until the user performs an operation to end thescanning of the medium with the pen scanner 2 (S13, No in S14).

When the user performs an operation to end the scanning of the mediumwith the pen scanner 2 (Yes in S14), the image acquisition means 20 endsthe scanning of the medium. Specifically, when the pen scanner 2 is putinto the pen-up state, the image acquisition means 20 stops thegeneration and holding of the image information.

The path calculation means 21 calculates the moving path of the penscanner 2 based on the plurality of pieces of image information held bythe image acquisition means 20 (S15). The vector calculation means 22calculates a plurality of vectors in the moving path of the pen scanner2 based on the moving path of the pen scanner 2 calculated by the pathcalculation means 21, and generates a plurality of pieces of vectorinformation presenting the plurality of calculated vectors. Theprocessing method determination means 30 of the smartphone 3 determineswhether the moving path of the pen scanner 2 is linear or not based onthe plurality of vectors presented by the plurality of pieces of vectorinformation generated by the vector calculation means 22 (S16).

When the processing method determination means 30 determines whether themoving path of the pen scanner 2 is linear (Yes in S16), the firstselection means 31 selects the first processing means 32 as a processingmeans that processes the plurality of pieces of image informationgenerated by the image acquisition means 20. The first processing means32 combines the images presented by the plurality of pieces of imageinformation (S17) and performs OCR on a character sequence of a pagenumber contained in the image obtained by the combining (S18). The firstprocessing means 32 thereby generates text information that presents, inthe form of an electronic text, the character sequence of the pagenumber contained in the image obtained by the combining. The firstprocessing means 32 adds attribute information indicating the attribute“page number” to the generated text information. The text information isthereby generated as page number information indicating a page number bythe first processing means 32.

The second selection means 34 acquires the page number informationgenerated by the first processing means 32. The information controlmeans 40 stores the page number information acquired by the secondselection means 34 into the storage means 41 (S19). After that, thescanning system 1 restarts the operation from Step S11.

When the processing method determination means 30 determines whether themoving path of the pen scanner 2 is non-linear (No in S16), the firstselection means 31 selects the second processing means 33 as aprocessing means that processes the plurality of pieces of imageinformation generated by the image acquisition means 20. The secondprocessing means 33 combines the images presented by the plurality ofpieces of image information (S20) and generates image informationpresenting the image obtained by the combining. The second processingmeans 33 compresses the generated image information and therebygenerates compressed image information presenting the compressed image(S21). For example, the second processing means 33 converts anuncompressed file presenting the image obtained by the combining into aJPG file. The second processing means 33 adds attribute informationindicating the attribute “body” to the generated compressed imageinformation. The compressed image information is thereby generated asbody information presenting a body by the second processing means 33.

The second selection means 34 acquires the body information generated bythe second processing means 33. The information control means 40 storesthe body information acquired by the second selection means 34 into thestorage means 41 (S22). After that, the scanning system 1 restarts theoperation from Step S11.

The effects of the first embodiment are described hereinafter withreference to FIG. 10. In the scanning system 1 according to the firstembodiment, when a user scans a medium by linearly moving the penscanner 2, it is possible to convert a character string shown on thescanned part of the medium into an electronic text and store it into theserver 4.

Further, in the scanning system 1 according to the first embodiment,when a user scans a medium by non-linearly moving the pen scanner 2, itis possible to convert a character string or a drawing shown on thescanned part of the medium into an electronic image, compress thisimage, and store it into the server 4.

In this configuration, when a user wishes to scan a page number of apaper book, the user scans it by linearly moving the pen scanner 2, andwhen a user wishes to scan a body of a paper book, the user scans it bynon-linearly moving the pen scanner 2 as shown in FIG. 10, for example,and it is thereby possible to store the page number and the body indifferent forms from each other. Specifically, when a user wishes tostore a page number in the form of an electronic text and store a bodyin the form of an electronic image, it is possible to store the pagenumber and the body respectively in appropriate forms.

As described above, according to the first embodiment, (1) acquisitionof information, (2) conversion of and attribute addition to acquiredinformation and (3) storage can be done in a single action that scans amedium with the pen scanner 2. This eliminates the need for an operationthat specifies the type (attribute) of information before scanning theinformation in a medium, for example. In other words, there is no needfor operations other than scanning, thereby enhancing the convenience.Thus, in the scanning system 1 according to the first embodiment, it ispossible to store different types of information on a medium to bescanned in appropriate forms with a simple operation.

Although an example where the calculation of the moving path of the penscanner 2 and the vectors is performed in the pen scanner 2 is describedabove, it is not limited thereto. For example, the smartphone 3 mayperform the calculation of the moving path of the pen scanner 2 and thevectors. Specifically, the smartphone 3 may include the path calculationmeans 21 and the vector calculation means 22.

Further, although an example where the determination of a processingmethod of image information and the processing of the image informationby the determined processing method are performed in the smartphone 3 isdescribed above, it is not limited thereto. For example, the pen scanner2 may perform the determination of a processing method of imageinformation and the processing of the image information by thedetermined processing method. Specifically, the pen scanner 2 mayinclude the means 30 to 34, which the smartphone 3 includes in the abovedescription. In this case, the smartphone 3 operates as a device thatrelays the text information and the compressed image informationgenerated by the pen scanner 2 to the server 4. Further, in this case,the text information and the compressed image information may be storedin the pen scanner 2, rather than being transmitted to the server 4.

Further, all of the calculation of the moving path of the pen scanner 2and the vectors, the determination of a processing method of imageinformation, and the processing of the image information by thedetermined processing method may be performed in the server 4.Specifically, the server 4 may include the means 21, 22, 30 to 34described above. In this case, the smartphone 3 operates as a devicethat relays the image information generated by the pen scanner 2 to theserver 4.

Furthermore, although an example where the text information and thecompressed image information are stored in the server 4 is describedabove, it is not limited thereto. For example, the text information andthe compressed image information may be stored in the smartphone 3,rather than being transmitted to the server 4.

As described above, according to the first embodiment, as shown in theschematic configuration of FIG. 36, an image acquisition unit 91 (whichcorresponds to the image acquisition means 20) acquires a plurality ofpieces of image information generated by continuously scanning a mediumto be scanned. A path calculation unit 92 (which corresponds to the pathcalculation means 21) calculates a path of scanning the medium based onthe plurality of pieces of image information acquired by the imageacquisition unit 91. A processing method determination unit 93 (whichcorresponds to the processing method determination means 30) determinesa processing method of the plurality of pieces of image information inaccordance with the path calculated by the path calculation unit 92.Then, a processing unit 94 (which corresponds to the first processingmeans 32 and the second processing means 33) processes the plurality ofpieces of image information by the processing method determined by theprocessing method determination unit 93 and thereby converts theplurality of pieces of image information into information in the formscorresponding to the processing method.

In this configuration, by an operation of scanning a medium, aconversion method of image information obtained by the scanning can bespecified. Therefore, according to the first embodiment, it is possibleto store different types of information on a medium to be scanned inappropriate forms with a simple operation.

Second Embodiment

A second embodiment is described hereinafter. In the followingdescription, the same matter as in the first embodiment described aboveis denoted by the same reference symbol or the like, and the descriptionthereof is omitted as appropriate. The schematic configuration of thescanning system 1 according to the second embodiment is the same as theschematic configuration of the scanning system 1 according to the firstembodiment and thus not redundantly described.

The hardware configuration of the pen scanner 2 according to the secondembodiment is described hereinafter with reference to FIG. 11. The penscanner 2 according to the second embodiment is different from the penscanner 2 according to the first embodiment in that it further includesan operating state switch button 16.

The operating state switch button 16 is an input device on which a userperforms an operation to switch the operating state of the pen scanner2. The operating state switch button 16 generates a notification signalindicating whether it is pressed by a user or not, and outputs it to theMCU 13. When pressed, the operating state switch button 16 generates anotification signal indicating that it is pressed, and outputs it to theMCU 13. When not pressed, the operating state switch button 16 generatesa notification signal indicating that it is not pressed, and outputs itto the MCU 13.

When the notification signal indicating that the operating state switchbutton 16 is pressed is input from the operating state switch button 16at the time of scanning a medium, the MCU 13 transmits, to thesmartphone 3, notification information indicating that the operatingstate of the pen scanner 2 at the time of scanning a medium is a firststate, together with a plurality of pieces of image information and aplurality of pieces of vector information. On the other hand, when thenotification signal indicating that the operating state switch button 16is not pressed is input from the operating state switch button 16 at thetime of scanning a medium, the MCU 13 transmits, to the smartphone 3,notification information indicating that the operating state of the penscanner 2 at the time of scanning a medium is a second state, togetherwith a plurality of pieces of image information and a plurality ofpieces of vector information.

A method of determining a processing method according to the secondembodiment is described hereinafter with reference to FIG. 12.

In the first embodiment, the type of information that is input byscanning is specified based on whether the moving path of the penscanner 2 is linear or not when scanning a medium. On the other hand, inthe second embodiment, the type of information that is input by scanningcan be specified also based on whether the operating state switch button16 is pressed or not (whether the pen scanner 2 is the first state orthe second state) when scanning a medium. In other words, in the secondembodiment, four types of information can be specified as the type ofinformation that is input by scanning by the combination of whether themoving path of the pen scanner 2 is linear or not and whether theoperating state switch button 16 is pressed or not as shown in FIG. 12.

Specifically, the scanning system 1 according to the second embodimentdetermines the type of information that is input by scanning asdescribed in the following (1) to (4).

(1) When the moving path of the pen scanner 2 is linear and theoperating state switch button 16 is pressed (i.e. the operating state ofthe pen scanner 2 is the first operating state), the smartphone 3determines that the information that is input by scanning is a pagenumber.

(2) When the moving path of the pen scanner 2 is non-linear and theoperating state switch button 16 is pressed (i.e. the operating state ofthe pen scanner 2 is the first operating state), the smartphone 3determines that the information that is input by scanning is a drawing

(3) When the moving path of the pen scanner 2 is linear and theoperating state switch button 16 is not pressed (i.e. the operatingstate of the pen scanner 2 is the second operating state), thesmartphone 3 determines that the information that is input by scanningis a body.

(4) When the moving path of the pen scanner 2 is non-linear and theoperating state switch button 16 is not pressed (i.e. the operatingstate of the pen scanner 2 is the second operating state), thesmartphone 3 determines that the information that is input by scanningis notes.

Note that the correspondence between the combination of whether themoving path of the pen scanner 2 is linear or not and whether theoperating state switch button 16 is pressed or not and the type ofinformation that is input by scanning is not limited to theabove-described example. For example, when the moving path of the penscanner 2 is linear and the operating state switch button 16 is pressed,information that is input by scanning may be a body, and when the movingpath of the pen scanner 2 is linear and the operating state switchbutton 16 is not pressed, information that is input by scanning may be apage number. For example, when the moving path of the pen scanner 2 isnon-linear and the operating state switch button 16 is pressed,information that is input by scanning may be notes, and when the movingpath of the pen scanner is non-linear and the operating state switchbutton 16 is not pressed, the information that is input by scanning maybe a drawing.

Further, in the second embodiment, when the calculation of vectors ends,the MCU 13 transmits path information stored in the memory 14, togetherwith a plurality of pieces of image information and a plurality ofpieces of vector information stored in the memory 14, to the smartphone3.

The detailed configuration of the scanning system 1 according to thesecond embodiment is described hereinafter with reference to FIG. 13.

As shown in FIG. 13, the pen scanner 2 according to the secondembodiment is different from the pen scanner 2 according to the firstembodiment in that it further includes a state setting means 23 and astate determination means 24.

The path calculation means 21 according to the second embodimenttransmits the generated path information to the smartphone 3.

With the state setting means 23, a user performs an operation to set thestate of the pen scanner 2. Stated differently, a user sets the state ofthe pen scanner 2 to the first state or the second state by the statesetting means 23. Specifically, the operating state switch button 16operates as the state setting means 23.

The state determination means 24 determines whether the operating stateof the pen scanner 2 is the first state or the second state based on theuser's operation on the state setting means 23. The state determinationmeans 24 generates notification information indicating whether theoperating state of the pen scanner 2 is the first state or the secondstate and transmits it to the smartphone 3. Specifically, the MCU 13operates as the state determination means 24.

Although the notification information is transmitted in association witha plurality of pieces of image information that have been generated frompen-down to pen-up in one scan as described above, the statedetermination means 24 may determine whether the operating state switchbutton 16 is pressed or not for the scan based on arbitrary criteria.For example, from pen-down to pen-up, the state determination means 24may generate the notification information indicating the first statewhen the operating state switch button 16 is pressed during pen-down,may generate the notification information indicating the first statewhen the operating state switch button 16 is pressed during pen-up, ormay generate the notification information indicating the first statewhen the operating state switch button 16 is continuously pressed frompen-down to pen-up. The state determination means 24 may generate thenotification information indicating the second state in the other cases.

As shown in FIG. 13, the smartphone 3 according to the second embodimentis different from the smartphone 3 according to the first embodiment inthat it further includes a third processing means 35 and a fourthprocessing means 36.

The processing method determination means 30 according to the secondembodiment is different from the processing method determination means30 according to the first embodiment in that it determines a processingmethod of a plurality of pieces of image information received from theimage acquisition means 20 of the pen scanner 2 or path informationbased on the notification information received from the statedetermination means 24 of the pen scanner 2 in addition to a pluralityof pieces of vector information received from the vector calculationmeans 22 of the pen scanner 2.

When the moving path of the pen scanner 2 is linear and the operatingstate of the pen scanner 2 is the first state, the processing methoddetermination means 30 determines to process a plurality of pieces ofimage information received from the pen scanner 2 by a first processingmethod performed by the first processing means 32. When the moving pathof the pen scanner 2 is non-linear and the operating state of the penscanner 2 is the first state, the processing method determination means30 determines to process a plurality of pieces of image informationreceived from the pen scanner 2 by a second processing method performedby the second processing means 33. When the moving path of the penscanner 2 is linear and the operating state of the pen scanner 2 is thesecond state, the processing method determination means 30 determines toprocess a plurality of pieces of image information received from the penscanner 2 by a third processing method performed by the third processingmeans 35. When the moving path of the pen scanner 2 is non-linear andthe operating state of the pen scanner 2 is the second state, theprocessing method determination means 30 determines to process pathinformation received from the pen scanner 2 by a fourth processingmethod performed by the fourth processing means 36.

The first selection means 31 receives a plurality of pieces of imageinformation transmitted from the image acquisition means 20 of the penscanner 2 and path information. The first selection means 31 outputs thereceived plurality of image information or path information to one ofthe first processing means 32, the second processing means 33, the thirdprocessing means 35 and the fourth processing means 36 in accordancewith a determination result of the processing method determination means30. To be specific, when the processing method determination means 30determines to process the plurality of pieces of image information orthe path information by the third processing method, the first selectionmeans 31 outputs the plurality of pieces of image information to thethird processing means 35. On the other hand, when the processing methoddetermination means 30 determines to process the plurality of pieces ofimage information or the path information by the fourth processingmethod, the first selection means 31 outputs the path information to thefourth processing means 36. The cases when the processing methoddetermination means 30 determines to process the plurality of pieces ofimage information by the first processing method and when the processingmethod determination means 30 determines to process the plurality ofpieces of image information by the second processing method are the sameas those described in the first embodiment.

As the third processing method, the third processing means 35 combines aplurality of images presented by the plurality of pieces of imageinformation, performs OCR on a character sequence contained in the imagegenerated by the combining, generates text information where thecharacter sequence is presented in the form of an electronic text, andadds an attribute to the generated text information.

As the fourth processing method, the fourth processing means 36generates an image presenting the moving path of the pen scanner 2indicated by the path information as notes, compresses the generatedimage, generates compressed image information presenting the compressedimage, and adds an attribute to the generated compressed imageinformation.

Note that the third processing means 35 also combines a plurality ofimages in the state where those images are superimposed on one anotherwith the positions of the feature point that is the same among theplurality of images being aligned. The reason is as described earlier.

The second selection means 34 acquires the text information or thecompressed image information generated by one of the first processingmeans 32, the second processing means 33, the third processing means 35and the fourth processing means 36 in accordance with a determinationresult of the processing method determination means 30. The secondselection means 34 transmits the acquired text information or compressedimage information to the server 4.

A CPU included in the smartphone 3, for example, executes a programstored in a storage means included in the smartphone 3, and therebyoperates as the third processing means 35 and the fourth processingmeans 36 in the same manner as the above-described means 30 to 34.

A scanning process of the scanning system 1 according to the secondembodiment is described hereinafter with reference to FIG. 14. In thefollowing description, an example where the scanning system 1 operatesas follows is described.

It is assumed that, when a user scans a page number of a paper book, theuser scans the page number by moving the pen scanner 2 in a linearmoving path while pressing the operating state switch button 16. It isalso assumed that, when a user scans a drawing of a paper book, the userscans the drawing by moving the pen scanner 2 in a non-linear movingpath while pressing the operating state switch button 16. It is alsoassumed that, when a user scans a body, the user scans the body bymoving the pen scanner 2 in a linear moving path without pressing theoperating state switch button 16. It is also assumed that, when a userenters notes, the user enters notes by moving the pen scanner 2 in anon-linear moving path without pressing the operating state switchbutton 16.

The processing steps S11 to S15 are the same as those described in thefirst embodiment. After the processing step S15, the processing methoddetermination means 30 determines whether the operating state of the penscanner 2 indicated by the notification information generated by thestate determination means 24 is the first state or the second state(S31). As described above, this notification information is informationindicating whether the state of the pen scanner 2 during scanning is thefirst state where the operating state switch button 16 is pressed or thesecond state where the operating state switch button 16 is not pressed.Stated differently, in Step S31, the processing method determinationmeans 30 determines whether the operating state of the pen scanner 2during scanning has been the first state or the second state.

When it is determined that the operating state of the pen scanner 2 isthe first state where the operating state switch button 16 is pressed(Yes in S31), the processing step S16 to S22 are the same as thosedescribed above in the first embodiment. In the second embodiment,however, the second processing means 33 adds attribute informationindicating the attribute “drawing” to the generated compressed imageinformation. The compressed image information is thereby generated asdrawing information presenting a drawing by the second processing means33.

When it is determined that the operating state of the pen scanner 2 isthe second state where the operating state switch button 16 is notpressed (No in S31), the processing method determination means 30determines whether the moving path of the pen scanner 2 is linear or notbased on a plurality of vectors presented by the plurality of pieces ofvector information generated by the vector calculation means 22 (S32).

When the processing method determination means 30 determines that themoving path of the pen scanner 2 is linear (Yes in S32), the firstselection means 31 selects the third processing means 35 as a processingmeans that processes the plurality of pieces of image informationgenerated by the image acquisition means 20. The third processing means35 combines images presented by the plurality of pieces of imageinformation (S33) and performs OCR on a character sequence of a bodycontained in the image obtained by the combining (S34). The thirdprocessing means 35 thereby generates text information that presents, inthe form of an electronic text, the character sequence of the bodycontained in the image obtained by the combining. The third processingmeans 35 adds attribute information indicating the attribute “body” tothe generated text information. The text information is therebygenerated as body information presenting a body by the third processingmeans 35.

The second selection means 34 acquires the body information generated bythe third processing means 35. The information control means 40 storesthe body information acquired by the second selection means 34 into thestorage means 41 (S35). After that, the scanning system 1 restarts theoperation from Step S11.

When the processing method determination means 30 determines whether themoving path of the pen scanner 2 is non-linear (No in S32), the firstselection means 31 selects the fourth processing means 36 as aprocessing means that processes the path information generated by thepath calculation means 21. The fourth processing means 36 generatesimage information presenting an image where the moving path indicated bythe path information is handwriting in note-taking (S36). The fourthprocessing means 36 compresses the generated image information andthereby generates compressed image information presenting the compressedimage (S37). For example, the fourth processing means 36 converts anuncompressed file presenting the image obtained by the combining into aJPG file. The fourth processing means 36 adds attribute informationindicating the attribute “notes” to the generated compressed imageinformation. The compressed image information is thereby generated asnote information presenting notes by the fourth processing means 36.

The second selection means 34 acquires the note information generated bythe fourth processing means 36. The information control means 40 storesthe note information acquired by the second selection means 34 into thestorage means 41 (S38). After that, the scanning system 1 restarts theoperation from Step S11.

Although an example where the note information is image information isdescribed above, it is not limited thereto. For example, the noteinformation may be vector data (coordinate information) indicating thepositions of the starting and ending points of each line in the movingpath (notes). In other words, the note information may be information inany form as long as it is information that presents the path of scanninga medium as notes.

Further, although an example where the setting of the operating state isperformed in the pen scanner 2 is described above, it is not limitedthereto. For example, the smartphone 3 may perform the setting of theoperating state. Specifically, the smartphone 3 may include the statesetting means 23 and the state determination means 24. In this case, aninput means (for example, a touch panel) of the smartphone 3 operates asthe state setting means 23.

As described above, the scanning system 1 according to the secondembodiment is different from the scanning system 1 according to thefirst embodiment in that it includes the operating state switch button16 that is pressed by a user. Further, the processing methoddetermination means 30 according to the second embodiment is differentfrom the processing method determination means 30 according to the firstembodiment in that it determines to perform different processing methodsdepending further on whether the button is pressed or not when scanninga medium.

In this configuration, it is possible to scan a medium by specifying alarger number of types of information and perform conversion with asimple operation that a user performs scanning while pressing a button.In other words, in the scanning system 1 according to the secondembodiment, it is possible to store a larger number of different typesof information in appropriate forms with a simple operation.

Further, a plurality of processing methods prepared in the secondembodiment include the first processing method to the fourth processingmethod. The first processing method generates information afterconversion of a plurality of pieces of image information as page numberinformation presenting a page number of a paper book scanned as amedium. The second processing method generates information afterconversion of a plurality of pieces of image information as drawinginformation presenting a drawing of a paper book scanned as a medium.The third processing method generates information after conversion of aplurality of pieces of image information as body information presentinga body of a paper book scanned as a medium. The fourth processing methodgenerates information after conversion of a plurality of pieces of imageinformation as note information presenting a path of scanning a mediumcalculated based on the plurality of pieces of image information asnotes.

In this configuration, it is possible to store different types ofinformation contained in a paper book and notes concerning them in adistinguishable manner.

Modified Example of Second Embodiment

In the above-described second embodiment, in the case where theoperating state switch button 16 is pressed when the pen scanner 2 is inthe pen-up state between two scans of a body, the input may be processedas the insertion of a line break, and in the case where the operatingstate switch button 16 is not pressed instantaneously between two scansof a body, the two bodies may be joined without a line break.

For example, in the case where an operation is performed on the statesetting means 23 (when the operating state switch button 16 is pressed)during the period from pen-up to pen-down of the pen scanner 2, thestate determination means 24 generates notification informationindicating the insertion of a line break and transmits it to thesmartphone 3.

In the case where notification information indicating the insertion of aline break is received from the pen scanner 2 when body information isacquired from the third processing means 35, the second selection means34 of the smartphone 3 adds the notification information indicating theinsertion of a line break to the body information and transmits them tothe server 4. On the other hand, in the case where notificationinformation indicating the insertion of a line break is not receivedfrom the pen scanner 2 when body information is acquired from the thirdprocessing means 35, the second selection means 34 of the smartphone 3transmits the body information to the server 4 without adding thenotification information indicating the insertion of a line break.

When the notification information indicating the insertion of a linebreak is given to the body information received from the smartphone 3,the information control means 40 of the server 4 updates the bodyinformation stored in the storage means 41 to information that presentsa body in which a body presented by the stored body information, anewline character, and a body presented by the received body informationare joined. On the other hand, when the notification informationindicating the insertion of a line break is not given to the bodyinformation received from the smartphone 3, the information controlmeans 40 of the server 4 updates the body information stored in thestorage means 41 to information that presents a body joining the bodypresented by that stored body information and the body presented by thereceived body information without inserting a newline character betweenthem.

Third Embodiment

A third embodiment is described hereinafter. In the followingdescription, the same matter as in the second embodiment described aboveis denoted by the same reference symbol or the like, and the descriptionthereof is omitted as appropriate. The schematic configuration of thescanning system 1 and the hardware configuration of the pen scanner 2according to the third embodiment are the same as the schematicconfiguration of the scanning system 1 and the hardware configuration ofthe pen scanner 2 according to the second embodiment and thus notredundantly described.

In the third embodiment, drawing information and body information arestored in association with page number information generated mostrecently. Further, note information is stored in association with a pairof page number information generated most recently and body informationgenerated most recently. Thus, if a user scans a page number and thenscans a drawing and a body on a page with that page number, it ispossible to easily find, from the page number information, on which pagein a paper book the drawing presented by the drawing information and thebody presented by the body information stored in the server 4 are shown.Further, if a user scans a page number and a body and then enters notesabout the body, it is possible to easily find, from the page numberinformation and the body information, which body on which page in apaper book the notes presented by the note information are taken about.

The detailed configuration of the scanning system 1 according to thethird embodiment is described hereinafter with reference to FIG. 15.Note that, in FIG. 15, a storage means included in the smartphone 3 isexplicitly shown as “storage means 39”.

As shown in FIG. 15, the server 4 according to the third embodiment isdifferent from the server 4 according to the second embodiment in thatit includes a structuring means 42 in place of the information controlmeans 40.

In the third embodiment, when the second selection means 34 transmitspage number information to the server 4, it temporarily stores the pagenumber information into the storage means 39. Further, when the secondselection means 34 transmits body information to the server 4 also, ittemporarily stores the body information into the storage means 39. Thus,the page number information and the body information generated mostrecently are stored in the storage means 39.

Then, when the second selection means 34 transmits drawing informationto the server 4, it adds the page number information stored in thestorage means 39 to the drawing information and then transmits them tothe server 4. Likewise, when the second selection means 34 transmitsbody information to the server 4, it adds the page number informationstored in the storage means 39 to the body information and thentransmits them to the server 4. Further, when the second selection means34 transmits note information to the server 4, it adds the page numberinformation and the body information stored in the storage means 39 tothe note information and then transmits them to the server 4.

The structuring means 42 receives the page number information, the bodyinformation, the drawing information and the note informationtransmitted from the second selection means 34 of the smartphone 3 andstores them into the storage means 41. When the structuring means 42stores the body information into the storage means 41, it stores thebody information in association with the page number informationindicating the same page number as the page number information added tothe body information, out of the page number information stored in thestorage means 41. This is the same for the drawing information. Further,when the structuring means 42 stores the note information into thestorage means 41, it stores the note information into the storage means41 in association with the page number information and the bodyinformation indicating the same page number and the body as the pagenumber information and the body information added to the noteinformation, out of the page number information and the body informationstored in the storage means 41.

A scanning process of the scanning system 1 according to the thirdembodiment is described hereinafter with reference to FIG. 16.

The processing steps S11 to S21 are the same as those in the secondembodiments. Note that, however, in the third embodiment, the secondselection means 34 stores the page number information generated by thefirst processing means 32 into the storage means 39 between theprocessing step S18 and the processing step S19. Specifically, the pagenumber scanned most recently is updated to the page number of thecurrently scanned page (the currently read page) (S41).

On the other hand, after the processing step S21, the second selectionmeans 34 acquires the drawing information generated by the secondprocessing means 33 and the page number information stored in thestorage means 39 (S42). The structuring means 42 associates the pagenumber information that matches the page number information acquired bythe second selection means 34, out of the page number information storedin the storage means 41, with the drawing information acquired by thesecond selection means 34 and stores them into the storage means 41(S43).

The processing steps S32 to S34, S36 and S37 are the same as thosedescribed in the second embodiment. After the processing step S34, thesecond selection means 34 stores the body information generated by thethird processing means 35 into the storage means 39. Specifically, thebody scanned most recently is updated to the body scanned in thecurrently scanned page (the body read in the currently read page) (S44).

The second selection means 34 acquires the body information generated bythe second processing means 33 and the page number information stored inthe storage means 39 (S45). The structuring means 42 associates the pagenumber information that matches the page number information acquired bythe second selection means 34, out of the page number information storedin the storage means 41, with the body information acquired by thesecond selection means 34 and stores them into the storage means 41(S46).

On the other hand, after the processing step S37, the second selectionmeans 34 acquires the note information generated by the fourthprocessing means 36, and the page number information and the bodyinformation stored in the storage means 39 (S47). The structuring means42 associates a pair of page number information and body informationthat match a pair of the page number information and the bodyinformation acquired by the second selection means 34, out of pairs ofthe page number information and the body information stored in thestorage means 41, with the note information acquired by the secondselection means 34, and stores them into the storage means 41 (S48).

According to the above process, by scanning a body (170) after scanninga page number (171), it is possible to store page number informationindicating that page number (171) in association with body informationpresenting that body (170) as shown in FIG. 17. Further, as shown inFIG. 17, by scanning a drawing (172) after scanning a page number (171),it is possible to store page number information indicating that pagenumber (171) in association with drawing information presenting thatdrawing (172). It is thus possible to record on which page the scannedbody and drawing are shown.

Further, as shown in FIG. 17, by entering notes (173) after scanning apage number (171) and a body (170), it is possible to store page numberinformation and body information presenting those page number (171) andbody (170) in association with note information presenting the notes(173). It is thus possible to record for which body on which page theentered notes are taken.

Although an example of storing note information in association with apair of page number information generated most recently and bodyinformation generated most recently is described above, it is notlimited thereto. For example, note information may be stored inassociation with a pair of page number information generated mostrecently and drawing information generated most recently. In this case,after the processing step S21, the second selection means 34 may storethe drawing information generated by the second processing means 33 intothe storage means 39. As the processing step S47, the second selectionmeans 34 may acquire the page number information and the drawinginformation stored in the storage means 39 together with the noteinformation generated by the fourth processing means 36. Then, as theprocessing step S48, the structuring means 42 may associate a pair ofthe page number information and the drawing information that match apair of the page number information and the drawing information acquiredby the second selection means 34, out of pairs of the page numberinformation and the drawing information stored in the storage means 41,with the note information acquired by the second selection means 34, andstore them into the storage means 41. In this case, the processing stepS44 is not needed.

Further, both of the storing of note information in association with apair of page number information and body information and the storing ofnote information in association with a pair of page number informationand drawing information may be performed. In this case, both of theprocessing of the step S44 and the storing of drawing information intothe storage means 39 after the step S21 are performed. As the processingstep S47, the second selection means 34 may acquire information (bodyinformation or drawing information) generated most recently out of bodyinformation generated most recently and drawing information generatedmost recently. Then, as the processing step S48, the structuring means42 may associate a pair of page number information and body informationor drawing information that match a pair of the page number informationand the body information or the drawing information acquired by thesecond selection means 34 with the note information acquired by thesecond selection means 34, and store them into the storage means 41.

As described above, the scanning system 1 according to the thirdembodiment includes the structuring means 42 that stores, into thestorage means 41, the page number information generated most recently inassociation with the drawing information, the body information or thenote information generated after the generation of the page numberinformation. Further, the structuring means 42 stores, into the storagemeans 41, a pair of the page number information generated most recentlyand the body information or the drawing information generated mostrecently in association with the note information generated after thegeneration of the page number information and the body information.

In this configuration, because data scanned by a user are stored inassociation with each other, it is possible to facilitate the use ofinformation when the user searches for or refers to the informationstored in the server 4.

Fourth Embodiment

A fourth embodiment is described hereinafter. In the followingdescription, the same matter as in the third embodiment described aboveis denoted by the same reference symbol or the like, and the descriptionthereof is omitted as appropriate. The schematic configuration of thescanning system 1 and the hardware configuration of the pen scanner 2according to the fourth embodiment are the same as the schematicconfiguration of the scanning system 1 and the hardware configuration ofthe pen scanner 2 according to the third embodiment and thus notredundantly described.

In the fourth embodiment, when a non-linear path is detected as themoving path of the pen scanner 2, it is determined whether the movingpath corresponds to a predefined gesture. When the moving path of thepen scanner 2 corresponds to a gesture, a processing corresponding tothat gesture is performed. As the gesture, an arbitrary symbol (circle,square, triangle etc.), an alphabet or the like may be defined, forexample.

The detailed configuration of the scanning system 1 according to thefourth embodiment is described hereinafter with reference to FIG. 18.

As shown in FIG. 18, the smartphone 3 according to the fourth embodimentis different from the smartphone 3 according to the third embodiment inthat it further includes a referring means 37, a gesture definition DB38, a first sub-processing means 50, a second sub-processing means 51,and a third sub-processing means 52.

When the moving path of the pen scanner 2 is non-linear and theoperating state of the pen scanner 2 is the second state, the processingmethod determination means 30 according to the fourth embodiment causesthe referring means 37 to determine whether the moving path correspondsto a specified gesture.

When the moving path of the pen scanner 2 is non-linear and theoperating state of the pen scanner 2 is the second state, the referringmeans 37 determines whether the moving path of the pen scanner 2indicated by the path information received from the pen scanner 2corresponds to any of gestures indicated by a plurality of pieces ofgesture information contained in the gesture definition DB 38.

When the referring means 37 determines that the moving path of the penscanner 2 corresponds to any of gestures indicated by the plurality ofpieces of gesture information, it causes one of the first sub-processingmeans 50, the second sub-processing means 51 and the thirdsub-processing means 52 which corresponds to the gesture to performprocessing.

Specifically, in the third embodiment, as the plurality of pieces ofgesture information, three pieces of gesture information respectivelycorresponding to the first sub-processing means 50 to the thirdsub-processing means 52 are contained in the gesture definition DB 38.Note that, although an example where the number of the gestureinformation and the sub-processing means is three is described in thethird embodiment, it is not limited thereto.

The gesture definition DB 38 contains a plurality of pieces of gestureinformation as described above. The gesture definition DB 38 is storedin the storage means 39 of the smartphone 3. The gesture information isinformation indicating the template of the moving path which is definedas a gesture.

For example, when the similarity between the moving path of the penscanner 2 indicated by the path information received from the penscanner 2 and the template of the moving path indicated by the gestureinformation is equal to or more than a specified threshold, thereferring means 37 determines that the moving path of the pen scanner 2corresponds to the gesture indicated by the gesture information. On theother hand, when the similarity between the moving path of the penscanner 2 indicated by the path information received from the penscanner 2 and the template of the moving path indicated by the gestureinformation is less than a specified threshold, the referring means 37determines that the moving path of the pen scanner 2 does not correspondto the gesture indicated by the gesture information. Note that, becauseanother method may be used as a method of determining the similaritybetween the moving path and the gesture, the detailed descriptionthereof is omitted.

Each of the first sub-processing means 50, the second sub-processingmeans 51 and the third sub-processing means 52 performs specifiedprocessing. Arbitrary processing may be predetermined as the processingto be performed by each of the first sub-processing means 50 to thethird sub-processing means 52. For example, as the processing to beperformed by each of the first sub-processing means 50 to the thirdsub-processing means 52, processing to change the operation mode of thescanning system 1 may be defined.

For example, as the processing of the first sub-processing means 50,processing to switch the operation mode of the scanning system 1 between“reading mode” and “dictionary mode” may be performed. Note that, in“reading mode”, the scanning system 1 performs an operation to convertand store information scanned by a user who is reading a paper book asdescribed above. In this case, when a gesture corresponding to the firstsub-processing means 50 is input by a user as the moving path of the penscanner 2, the first sub-processing means 50 changes the operation modeof the scanning system 1 from “reading mode” to “dictionary mode”. Next,when the user scans a word which he/she does not understand in theoperation of scanning a body, a word indicated by the text informationobtained by the third processing means 35 is searched for from adictionary DB (not shown) included in the smartphone 3. For example, thedictionary DB is prestored in the storage means 39 of the smartphone 3.The dictionary DB is information where a plurality of words and themeanings of the plurality of words are respectively associated with eachother. The smartphone 3 displays the meaning corresponding to thesearched word in the dictionary DB on a display means (not shown)included in the smartphone 3. The display means is a touch panel, forexample. After that, when a gesture corresponding to the firstsub-processing means 50 is input again by the user as the moving path ofthe pen scanner 2, the first sub-processing means 50 changes theoperation mode of the scanning system 1 from “dictionary mode” to“normal mode”.

Note that the dictionary DB is not limited to be included in thesmartphone 3 as described above. For example, it may be stored in thestorage means 41 of the server 4. In this case, the smartphone 3transmits the text information obtained by the third processing means 35and directs the server 4 to make a search. The server 4 search thedictionary DB for the word indicated by the text information receivedfrom the smartphone 3, and transmits information indicating the meaningcorresponding to the searched word to the smartphone 3. Then, thesmartphone 3 may display the meaning indicated by the informationreceived from the server 4 on a display means.

A CPU included in the smartphone 3, for example, executes a programstored in the storage means 39 included in the smartphone 3, and therebyoperates as the referring means 37 and the sub-processing means 50 to 52in the same manner as the above-described means 30 to 36.

A scanning process of the scanning system 1 according to the fourthembodiment is described hereinafter with reference to FIG. 19. In thefollowing description, an example where the scanning system 1 operatesas follows is described.

The processing steps S11 to S21, S31 to S34, S36, S37 and S41 to S48 arethe same as those of the third embodiment. When the processing methoddetermination means 30 determines that the moving path of the penscanner 2 is non-linear (Non in S32), the referring means 37 refers tothe gesture definition DB 38 (S51), and determines whether the movingpath of the pen scanner 2 indicated by the path information matches anyof gestures indicated by a plurality of pieces of gesture informationcontained in the gesture definition DB 38 (S52).

When it is determined that the moving path of the pen scanner 2indicated by the path information matches any of gestures indicated by aplurality of pieces of gesture information (Yes in S52), onesub-processing means, out of the first sub-processing means 50, thesecond sub-processing means 51 and the third sub-processing means 52,which corresponds to the gesture that is determined to correspond to themoving path of the pen scanner 2 performs processing (S53). On the otherhand, when it is determined that the moving path of the pen scanner 2indicated by the path information does not match any of gesturesindicated by a plurality of pieces of gesture information (No in S52),the processing after the step S36 is performed.

Note that, as described above, the determination of a processing methodof image information and the processing of the image information by thedetermined processing method may be performed in the pen scanner 2 orthe server 4 and, in this case, the above-described processingconcerning a gesture may be performed also in the pen scanner 2 or theserver 4 that performs the determination of a processing method of imageinformation and the processing of the image information by thedetermined processing method. Specifically, the means 37, 38, 50 to 52concerning a gesture may be included in the pen scanner 2 or the server4.

Further, although an example where whether the moving path correspondsto a specified gesture or not is determined when the moving path of thepen scanner 2 is non-linear and the operating state of the pen scanner 2is the second state is described above, it is not limited thereto. Forexample, the processing method determination means 30 may cause thereferring means 37 to determine whether the moving path corresponds to aspecified gesture or not when the moving path of the pen scanner 2 isnon-linear and the operating state of the pen scanner 2 is the firststate.

As described above, in the fourth embodiment, gesture informationindicating a gesture, which is the template of the path of scanning amedium, is stored in the storage means 39 of the smartphone 3. Then,when the referring means 37 determines that the path of scanning themedium corresponds to a gesture indicated by the gesture information, itperforms specified sub-processing (which corresponds to the processingperformed by each of the first sub-processing means 50, the secondsub-processing means 51 and the third sub-processing means 52).

In this configuration, a larger variety of processing (five or moretypes of processing in the fourth embodiment) can be performed by thescanning with the pen scanner 2. Further, in the fourth embodiment,processing that is performed frequently (processing corresponding toscanning of a page number, a body and a drawing, and entering of notes)in the application of the scanning system 1 is assigned as processingthat can be performed with a simple operation. On the other hand,processing that is performed less frequently (change of an operationmode) may be assigned as sub-processing that corresponds to a gesture,where the operation of the pen scanner 2 is slightly complicated. It isthereby possible to increase the variety of processing to be performedwithout degrading user convenience.

Fifth Embodiment

A fifth embodiment is described hereinafter. In the followingdescription, the same matter as in the fourth embodiment described aboveis denoted by the same reference symbol or the like, and the descriptionthereof is omitted as appropriate. The schematic configuration of thescanning system 1 and the hardware configuration of the pen scanner 2according to the fifth embodiment are the same as the schematicconfiguration of the scanning system 1 and the hardware configuration ofthe pen scanner 2 according to the fourth embodiment and thus notredundantly described.

In the fifth embodiment, page number information, drawing information,body information and note information are stored in association withISBN information indicating ISBN (International Standard Book Number)specified by a user most recently. Thus, if a user specifies ISBN andthen scans a page number, a drawing and a body of a paper book specifiedby the ISBN, it is possible to easily find, from the ISBN information,in which paper book the page number, the drawing and the body presentedby the page number information, the drawing information and the bodyinformation stored in the server 4 are shown. Further, if a userspecifies ISBN and then enters notes, it is possible to easily find,from the ISBN information, for which paper book the notes are taken.

The detailed configuration of the scanning system 1 according to thefifth embodiment is described hereinafter with reference to FIG. 20.

As shown in FIG. 20, the pen scanner 2 according to the fifth embodimentis different from the pen scanner 2 according to the fourth embodimentin that it further includes an ID storage means 25 and an IDtransmitting means 26.

The ID storage means 25 stores ID information that uniquely identifiesthe pen scanner 2. The ID storage means 25 is prestored in a nonvolatilememory (for example, flash memory) included in the MCU 13, for example.

The ID transmitting means 26 acquires the ID information stored in theID storage means 25 and transmits it to the smartphone 3.

As shown in FIG. 20, the smartphone 3 according to the fifth embodimentis different from the smartphone 3 according to the fourth embodiment inthat it further includes an input means 53, an information control means54, an ISBM DB 55, and a display means 56.

The input means 53 receives an input of arbitrary content from a user.In response to an input from a user, the input means 53 generates aninput signal indicating the input content and outputs it to theinformation control means 54.

The information control means 54 acquires and holds the ID informationtransmitted from the pen scanner 2. Further, when a user inputs ISBN tothe input means 53, the information control means 54 generates and holdsISBN information indicating the ISBN input by the user based on theinput signal that is input from the input means 53 in response to theinput. For example, the information control means 54 holds the IDinformation and the ISBN information by storing them into the memory 14.

In the ISBM DB 55, a pair of ISBN information indicating ISBN and titleinformation indicating the title of a paper book identified by the ISBNindicated by the ISBN information is prestored for a plurality ofdifferent paper books. Note that the title information may furthercontain the edition of a paper book.

The display means 56 displays an image presenting arbitrary content. Thedisplay means 56 displays an image according to an instruction from theinformation control means 54, for example. A touch panel (not shown)included in the smartphone 3 operates as the input means 53 and thedisplay means 56.

A CPU included in the smartphone 3, for example, executes a programstored in the storage means 39 included in the smartphone 3, and therebyoperates as the information control means 54 in the same manner as theabove-described means 30 to 37 and 50 to 52.

Preprocessing of the scanning system 1 according to the fifth embodimentis described hereinafter with reference to FIG. 21.

When a CPU of the smartphone 3 starts running an application program, itstarts the operation as the means 30 to 37 and 50 to 54 of thesmartphone 3. The information control means 54 of the smartphone 3 waitsfor pairing information from the pen scanner 2 (S101).

When the ID transmitting means 26 of the pen scanner 2 is powered on andactivated, it transmits pairing information for pairing the pen scanner2 and the smartphone 3 to the smartphone 3 (S102). The pairinginformation contains ID information that is stored in the ID storagemeans 25.

The information control means 54 of the smartphone 3 receives thepairing information transmitted from the pen scanner (S103). Theinformation control means 54 forms a pair of the smartphone 3 and thepen scanner 2 based on the received pairing information. Further, theinformation control means 54 stores ID information contained in the pairinformation into the storage means 39. Specifically, ID indicated by thereceived ID information is stored as the ID of the pen scanner that iscurrently used for scanning (S104).

The information control means 54 gives the display means 56 aninstruction to display an image showing a book list. In response to theinstruction from the information control means 54, the display means 56displays the book list (S105). The book list is a list of pairs of ISBNand title of paper books that have been scanned by a user. Book listinformation presenting the book list is stored in the storage means 39of the smartphone 3. The information control means 54 generates an imageshowing the book list based on the book list information stored in thestorage means 39. The book list information is updated by theinformation control means 54.

When a paper book to be scanned is included in the book list displayedon the display means 56 (Yes in S106), a user inputs a selection of thepaper book from the book list to the input means 53 (S107). Theinformation control means 54 identifies the paper book selected by theuser from the book list based on the input signal that is input from theinput means 53 in response to the selection. The information controlmeans 54 acquires the ISBN of the identified paper book from the booklist and stores ISBN information indicating the acquired ISBN into thestorage means 39. Specifically, the ISBN that is input most recently isupdated to the ISBN of the paper book that is currently scanned (read)by the user (S108).

On the other hand, when a paper book to be scanned is not included inthe book list displayed on the display means 56 (No in S106), a userenters the ISBN of the paper book to be scanned (S109). The informationcontrol means 54 acquires the ISBN entered by the user based on theinput signal that is input from the input means 53 in response to theentering. The information control means 54 searches the ISBM DB 55 forthe acquired ISBN (S111, S112).

When the acquired ISBN is detected in the ISBM DB 55 (Yes in S112), thebook list is updated to add a pair of the acquired ISBN and the titlecorresponding to the ISBN in the ISBM DB 55 (S113).

On the other hand, when the acquired ISBN is not detected in the ISBM DB55 (No in S112), the information control means 54 gives the displaymeans 56 an instruction to display an image that prompts to enter atitle. In response to the instruction from the information control means54, the display means 56 displays an image that prompts to enter a title(S114).

A user enters a title of a paper book to be scanned to the input means53 (S115). The information control means 54 acquires the title enteredby the user based on the input signal that is input from the input means53 in response to the entering. The information control means 54 updatesthe book list to add a pair of the acquired ISBN and the acquired title(S113).

After the step S113, the information control means 54 stores ISBNinformation indicating the acquired ISBN into the storage means 39.Specifically, the ISBN that is input most recently is updated to theISBN of the paper book that is currently scanned (read) by the user(S108).

The information control means 54 transmits, to the pen scanner 2,notification information that notifies the completion of setting of apaper book to be scanned (S115). In response to receiving thenotification information from the smartphone 3, the image acquisitionmeans 20 of the pen scanner 2 waits for scanning of a page number(S116). This corresponds to the step S11 in the scanning processdescribed below by reference to FIG. 22.

A scanning process of the scanning system 1 according to the fifthembodiment is described hereinafter with reference to FIG. 22.

The scanning process of the scanning system 1 according to the fifthembodiment is different from the scanning process of the scanning system1 according to the fourth embodiment in that it has the steps S61, S62,S63, S64, S65 and S66 instead of the steps S42, S43, S45, S46, S47 andS48, respectively.

After the processing step S21, the second selection means 34 acquiresthe drawing information generated by the second processing means 33 andthe ID information, the ISBN information and the page number informationstored in the storage means 39 (S61). The structuring means 42associates a set of ID information, ISBN information and page numberinformation that matches a set of the ID information, the ISBNinformation and the page number information acquired by the secondselection means 34, out of sets of ID information, ISBN information andpage number information stored in the storage means 41, with the drawinginformation acquired by the second selection means 34, and stores theminto the storage means 41 (S62).

After the processing step S44, the second selection means 34 acquiresthe body information generated by the third processing means 35 and theID information, the ISBN information and the page number informationstored in the storage means 39 (S63). The structuring means 42associates a set of ID information, ISBN information and page numberinformation that matches a set of the ID information, the ISBNinformation and the page number information acquired by the secondselection means 34, out of sets of the ID information, the ISBNinformation and the page number information stored in the storage means41, with the body information acquired by the second selection means 34,and stores them into the storage means 41 (S64).

After the processing step S37, the second selection means 34 acquiresthe note information generated by the fourth processing means 36 and theID information, the ISBN information, the page number information andthe body information stored in the storage means 39 (S65). Thestructuring means 42 associates a set of ID information, ISBNinformation, page number information and body information that matches aset of the ID information, the ISBN information, the page numberinformation and the body information acquired by the second selectionmeans 34, out of sets of the ID information, the ISBN information, thepage number information and the body information stored in the storagemeans 41, with the note information acquired by the second selectionmeans 34, and stores them into the storage means 41 (S66).

Note that, although the ISBN is input through the smartphone 3 in theabove description, it is not limited thereto. For example, the ISBN maybe input through the pen scanner 2. Specifically, the pen scanner 2 mayinclude the input means 53 and a part of the information control means54. In this case, the pen scanner 2 has, as the input means 53, anoperation button for inputting ISBN. Then, the information control means54 of the pen scanner 2 may generate ISBN information and transmit it tothe smartphone 3, and the information control means 54 of the smartphone3 may perform the same processing as above based on the ISBN informationreceived from the pen scanner 2.

According to the above process, by scanning a body (230) aftertransmitting ID (231) of the pen scanner 2 by pairing, entering ISBN(232) and scanning a page number (234), it is possible to store IDinformation, ISBN information and page number information presenting theID (231), the ISBN (232) and the page number (234) in association withbody information presenting the body (230) as shown in FIG. 23. Further,by scanning a drawing (233) after transmitting ID (231) of the penscanner 2 by pairing, entering ISBN (232) and scanning a page number(234), it is possible to store ID information, ISBN information and pagenumber information presenting the ID (231), the ISBN (232) and the pagenumber (234) in association with drawing information presenting thedrawing (233) as shown in FIG. 23. It is thus possible to record onwhich page in which paper book the scanned body and drawing are shown.

Further, as shown in FIG. 23, by entering notes (235) after transmittingID (231) of the pen scanner 2 by pairing, entering ISBN (232) andscanning a page number (234 and a body (230), it is possible to store IDinformation, ISBN information, page number information and bodyinformation presenting the ID (231), the ISBN (232), the page number(234) and the body (233) in association with note information presentingthe notes (235). It is thus possible to record for which body on whichpage in which paper book the entered notes are taken.

Although an example of storing note information in association with aset of ID information, ISBN information generated most recently, pagenumber information generated most recently, and body informationgenerated most recently is described above, it is not limited thereto.For example, note information may be stored in association with a set ofID information, ISBN information generated most recently, page numberinformation generated most recently, and drawing information generatedmost recently. In this case, after the processing step S21, the secondselection means 34 may store the drawing information generated by thesecond processing means 33 into the storage means 39. As the processingstep S65, the second selection means 34 may acquire the ID information,the ISBN information, the page number information and the drawinginformation stored in the storage means 39 together with the noteinformation generated by the fourth processing means 36. Then, as theprocessing step S66, the structuring means 42 may associate a set of IDinformation, ISBN information, page number information and drawinginformation that match a set of the ID information, the ISBNinformation, the page number information and the drawing informationacquired by the second selection means 34, out of sets of the IDinformation, the ISBN information, the page number information and thedrawing information stored in the storage means 41, with the noteinformation acquired by the second selection means 34, and store theminto the storage means 41. In this case, the processing step S44 is notneeded.

Further, both of the storing of note information in association with apair of page number information and body information and the storing ofnote information in association with a pair of page number informationand drawing information may be performed. In this case, both of theprocessing of the step S44 and the storing of drawing information intothe storage means 39 after the step S21 are performed. As the processingstep S65, information (body information or drawing information)generated most recently out of body information generated most recentlyand drawing information generated most recently may be acquired. Then,as the processing step S66, the structuring means 42 may associate a setof ID information, ISBN information, page number information and bodyinformation or drawing information that match a set of the IDinformation, the ISBN information, the page number information and thebody information or the drawing information acquired by the secondselection means 34 with the note information acquired by the secondselection means 34, and store them into the storage means 41.

As described above, in the fifth embodiment, an input that specifiesISBN of a paper book to be scanned as a medium is made to the inputmeans 53. The structuring means 42 stores, into the storage means 41,ISBN information indicating ISBN specified most recently and page numberinformation generated most recently in association with drawinginformation, body information or note information generated after thespecifying of the ISBN and the generation of the page numberinformation.

In this configuration, because a group of pieces of information scannedfor each of different paper books can be constructed, it is possible tofacilitate the use of information when searching or referring to theinformation stored in the server 4.

Further, in the fifth embodiment, the pen scanner 2 includes the IDstorage means 25 in which ID information that uniquely identifies thepen scanner 2 is prestored and the ID transmitting means 26 whichtransmits the ID information stored in the ID storage means 25. Thestructuring means 42 stores, into the storage means 41, ID informationtransmitted from the ID transmitting means 26, ISBN informationindicating ISBN specified most recently and page number informationgenerated most recently in association with drawing information, bodyinformation or note information generated after the receiving of the IDinformation, the specifying of the ISBN and the generation of the pagenumber information.

In this configuration, because a group of pieces of information scannedfor each of different pen scanners 2 (that is, for each of differentusers) can be further constructed, it is possible to facilitate the useof information when searching or referring to the information stored inthe server 4.

First Modified Example of Fifth Embodiment

In the above-described fifth embodiment, when changing a paper book tobe scanned, a user needs to change a device he/she operates from the penscanner 2 to the smartphone 3 and perform an operation to select a paperbook from a book list or enter ISBN on the smartphone 3. To save thosesteps, as the operation mode of the scanning system 1, “ISBN scan mode”which the system can transition to by inputting a gesture and in whichISBN can be input by scanning may be prepared as described below.

The detailed configuration of the scanning system 1 according to thefirst modified example of the fifth embodiment is described hereinafterwith reference to FIG. 24.

The scanning system 1 according to the first modified example of thefifth embodiment is different from the scanning system 1 according tothe fifth embodiment in that it further includes a fifth processingmeans 57.

The first sub-processing means 50 according to the fifth embodimentperforms processing to switch the operation mode of the scanning system1 between “reading mode” and “ISBN scan mode”.

When the operation mode of the scanning system 1 is “reading mode”, thesmartphone 3 causes one of the first processing means 32, the secondprocessing means 33, the third processing means 35 and the fourthprocessing means 36 which is determined by the processing methoddetermination means 30 to operate in response to receiving a pluralityof pieces of image information from the pen scanner 2 as describedabove. On the other hand, when the operation mode of the scanning system1 is “ISBN scan mode”, the smartphone 3 causes the fifth processingmeans 57 to operate in response to receiving a plurality of pieces ofimage information from the pen scanner 2.

The fifth processing means 57 combines a plurality of images presentedby the plurality of pieces of image information, performs OCR on acharacter sequence contained in the image generated by the combining,and generates text information where the character sequence is presentedin the form of an electronic text as ISBN information indicating ISBN.

A scanning process in “ISBN scan mode” of the scanning system 1according to the first modified example of the fifth embodiment isdescribed hereinafter.

The image acquisition means 20 of the pen scanner 2 waits for a user toperform an operation of scanning ISBN of a paper book as a medium withthe pen scanner 2 (S120 and No in S121). Specifically, the imageacquisition means 20 waits for the pen scanner 2 to enter the pen-downstate.

When the user performs an operation to scan a medium with the penscanner 2 (Yes in S121), the image acquisition means 20 starts thescanning of the medium (S122). The image acquisition means 20 repeatsthe generation and holding of image information presenting the scannedimages of the medium until the user performs an operation to end thescanning of the medium with the pen scanner 2 (S122, No in S123).

The fifth processing means 57 combines a plurality of images presentedby the plurality of pieces of image information generated by the imageacquisition means 20 (S124), and performs OCR on a character sequence ofISBN contained in the image generated by the combining (S125). The fifthprocessing means 57 thereby generates, as ISBN information, textinformation where the character sequence of ISBN contained in the imagegenerated by the combining is presented in the form of an electronictext.

The information control means 54 determines whether ISBN indicated bythe ISBN information generated by the fifth processing means 57 isincluded in a book list presented by the book list information stored inthe storage means (S126). When the ISBN indicated by the ISBNinformation is included in the book list (Yes in S126), the informationcontrol means 54 stores the ISBN information into the storage means 39.Specifically, the ISBN that is scanned most recently is updated to theISBN of the paper book that is currently scanned (read) by the user(S127).

On the other hand, when the ISBN indicated by the ISBN information isnot included in the book list (No in S126), the information controlmeans 54 searches the ISBM DB 55 for the ISBN indicated by the ISBNinformation (S129). The processing steps S130 to S132, S127, S133 andS134 performed after that are the same as the processing steps S113 toS115, S108, S115 and S116 described in the fifth embodiment, and thusnot redundantly described.

As described above, in the first modified example of the fifthembodiment, when the referring means 37 determines that the path ofscanning a medium corresponds to a gesture indicated by the gestureinformation, it performs sub-processing that changes the operation modeof the scanning system 1 from normal mode to ISBN scan mode. When theoperation mode of the scanning system 1 is ISBN scan mode, the fifthprocessing means 57 converts a plurality of pieces of image informationinto information in a specified form, and generates the convertedinformation as ISBN information indicating ISBN of a paper book scannedas the medium. The structuring means 42 stores, into the storage means41, ISBN information generated most recently and page number informationgenerated most recently in association with drawing information, bodyinformation or note information generated after the generation of theISBN information and the page number information.

In this configuration, it is possible to input ISBN and switch a paperbook to be scanned (book to be read) only by operating the pen scanner 2(input of a gesture and scan of ISBN). It is thereby possible to switchpaper books to be scanned (books to be read) with a more simpleoperation.

Second Modified Example of Fifth Embodiment

In the above-described fifth embodiment, in order to obtain moreaccurate scan results of a body, when the body is scanned, the body thathas been incorrectly recognized by OCR may be corrected on the basis ofscan results of other people or official electronic books.

The detailed configuration of the server 4 according to the secondmodified example of the fifth embodiment is described hereinafter withreference to FIG. 26.

As shown in FIG. 26, the server 4 according to the second modifiedexample of the fifth embodiment is different from the server 4 accordingto the fifth embodiment in that it further includes a referring means43, a storage means 44, and a book DB 45.

When the structuring means 42 according to the second modified exampleof the fifth embodiment receives a set of ID information, ISBNinformation, page number information and body information, it outputsthe ISBN information, the page number information and the bodyinformation to the referring means 43, and directs the referring means43 to correct the body presented by the body information.

The referring means 43 corrects the body presented by the bodyinformation that is output from the structuring means 42 based oninformation stored in the storage means 44 and the storage means 45.

In the storage means 44, information that is the same as the informationstored in the storage means 41 is stored for each of a plurality ofother people. Specifically, as a result that another person uses anotherpen scanner 2 and another pen scanner 2, ID information, ISBNinformation, page number information, body information, drawinginformation and note information are stored in the storage means 44 inthe same manner as described above.

In the storage means 45, a plurality of electronic books where aplurality of paper books are respectively presented in the form ofelectronic texts are stored. The electronic books are officialelectronic books where paper books are accurately converted intoelectronic texts. For example, the electronic books are provided bypublishers of paper books. An electronic book is associated with ISBNinformation indicating its ISBN. Further, each page of an electronicbook is associated with page number information indicating its pagenumber.

Each of the storage means 44 and the storage means 45 includes at leastone of storage devices such as a volatile memory, a hard disk, and aflash memory (nonvolatile memory), for example. The storage deviceincluded in each of the storage means 41, the storage means 44 and thestorage means 45 may be provided entirely or partly.

The referring means 43 makes a partial match search between a bodypresented by the body information output from the structuring means 42and a body on a page with a page number indicated by the page numberinformation output from the structuring means 42 in an electronic bookidentified by ISBN indicated by the ISBN information output from thestructuring means 42.

When those bodies match exactly, the referring means 43 does not correctthe body presented by the body information output from the structuringmeans 42. When those bodies match partially, the referring means 43corrects the body presented by the body information output from thestructuring means 42 so as to match the body of the electronic book.

On the other hand, when an electronic book identified by ISBN indicatedby the ISBN information output from the structuring means 42 is notstored in the storage means 45, the referring means 43 makes a partialmatch search between a body presented by the body information outputfrom the structuring means 42 and a body presented by the bodyinformation associated with the ISBN information and the page numberinformation that are the same as a pair of the ISBN information and thepage number information output from the structuring means 42, out of thebody information stored in the storage means 44. At this time, thereferring means 43 makes a partial match search among bodies presentedby body information of a plurality of other people.

When the body presented by the body information output from thestructuring means 42 (body presented by the own body information)exactly matches the whole of the bodies presented by body information ofa plurality of other people, the referring means 43 does not correct thebody presented by the body information output from the structuring means42. When the body presented by the own body information partiallymatches at least one of the bodies presented by the body information ofa plurality of other people, the referring means 43 corrects the bodypresented by the body information output from the structuring means 42so as to match the at least one of the bodies presented by the bodyinformation of other people. At this time, the body presented by the ownbody information is corrected so as to exactly match the body with thelargest number of matches based on majority rule.

When the body information is corrected by the referring means 43, thestructuring means 42 replaces the body information to be stored in thestorage means 41 with the corrected body information.

As described above, in the second modified example of the fifthembodiment, a plurality of electronic books officially generated from aplurality of paper books are stored in the storage means 45. When a bodypresented by the generated body information partially matches a body ona page with a page number indicated by page number information generatedmost recently in an electronic book corresponding to ISBN specified mostrecently, the referring means 43 corrects the body presented by thegenerated body information so as to exactly match the body on the pagewith the page number indicated by the page number information generatedmost recently.

First, in the second modified example of the fifth embodiment, aplurality of sets of ISBN information, page number information and bodyinformation generated when other users scan media are stored in thestorage means 44. When an electronic book corresponding to ISBNspecified most recently is not stored in the storage means 45, and abody presented by the generated body information partially matches atleast one of bodies presented by the body information of a plurality ofother people associated with the ISBN information and the page numberinformation that are the same as the ISBN information indicating theISBN specified most recently and the page number information generatedmost recently, the referring means 43 corrects the body presented by thegenerated body information so as to exactly match the body with thelargest number of matches among the bodies presented by the bodyinformation of the plurality of other people.

In this configuration, even when a body obtained by scanning isincorrectly converted by OCR, it is possible to correct it to the rightone.

Sixth Embodiment

A sixth embodiment is described hereinafter. In the followingdescription, the same matter as in the second embodiment described aboveis denoted by the same reference symbol or the like, and the descriptionthereof is omitted as appropriate. The schematic configuration of thescanning system 1 according to the sixth embodiment is the same as theschematic configuration of the scanning system 1 according to the firstembodiment and thus not redundantly described.

The hardware configuration of the pen scanner 2 according to the sixthembodiment is described hereinafter with reference to FIG. 27. The penscanner 2 according to the sixth embodiment is different from the penscanner 2 according to the second embodiment in that it further includesa display device 17.

The display device 17 is a device that displays a notification of aprocessing method determined by the processing method determinationmeans 30 of the smartphone 3. The display device 17 includes LEDsrespectively corresponding to a plurality of processing methods preparedin advance, and it may light up the LED corresponding to the processingmethod determined by the processing method determination means 30. Inthis case, the display device 17 includes four LEDs respectivelycorresponding to first to fourth processing methods. Further, thedisplay device 17 may be a display panel which displays an image thatnotifies a processing method determined by the processing methoddetermination means 30, such as a liquid crystal display panel or anorganic EL display panel, for example.

When a processing method is determined, the smartphone 3 according tothe sixth embodiment generates notification information indicating thedetermined processing method and transmits it to the pen scanner 2. TheMCU 13 of the pen scanner 2 displays, on the display device 17, anotification of the processing method indicated by the notificationinformation received from the smartphone 3 through the transceiver 15.

The detailed configuration of the scanning system 1 according to thesixth embodiment is described hereinafter with reference to FIG. 28.

As shown in FIG. 28, the pen scanner 2 according to the sixth embodimentis different from the pen scanner 2 according to the third embodiment inthat it further includes a display control means 27 and a display means28.

When a processing method for processing a plurality of pieces of imageinformation or path information is determined from the first processingmethod to the fourth processing method, the processing methoddetermination means 30 according to the sixth embodiment generatesnotification information indicating the determined processing method andtransmits it to the pen scanner 2.

The display control means 27 gives the display means 28 an instructionto display a notification of the processing method indicated by thenotification information received from the smartphone 3. Specifically,the MCU 13, the memory 14 and the transceiver 15 operate as the displaycontrol means 27.

In response to an instruction from the display control means 27, thedisplay means 28 displays a notification of the determined processingmethod. Specifically, the display device 17 operates as the displaymeans 28.

Note that a scanning process of the scanning system 1 according to thesixth embodiment is the same as a scanning process of the scanningsystem 1 according to the second embodiment except for notifying adetermined processing method to a user as described above, and theredundant description thereof is omitted.

Note that, although an example where the display control means 27 (thedisplay device 17) and the display means 28 are applied to the secondembodiment is described above, it is not limited thereto. The displaycontrol means 27 (the display device 17) and the display means 28 may beapplied to any of the first to fifth embodiments. Although notificationinformation indicating one of the first processing method and the secondprocessing method is notified in this case, that is obvious and notdescribed in detail.

As described above, in the sixth embodiment, the display control means27 displays, on the display means 28, a notification of a processingmethod determined by the processing method determination means 30.

In this configuration, it is possible to know whether scannedinformation is converted into information as a user intends. It is thuspossible to easily detect malfunction of the scanning system 1, forexample.

Seventh Embodiment

A seventh embodiment is described hereinafter. In the followingdescription, the same matter as in the fifth embodiment described aboveis denoted by the same reference symbol or the like, and the descriptionthereof is omitted as appropriate. The schematic configuration of thescanning system 1 and the hardware configuration of the pen scanner 2according to the seventh embodiment are the same as the schematicconfiguration of the scanning system 1 and the hardware configuration ofthe pen scanner 2 according to the fifth embodiment and thus notredundantly described.

In the seventh embodiment, the operation mode of the scanning system 1includes “reading mode” to scan information of a paper book and “viewingmode” to view information input by other people to a paper book in asocial reading service (or an online book shopping site). In thisconfiguration, a search can be easily made by simply scanninginformation of a paper book as a search key for searching informationinput by other people to a paper book in a social reading service.

The detailed configuration of the scanning system 1 according to theseventh embodiment is described hereinafter with reference to FIG. 29.

As shown in FIG. 29, the server 4 according to the seventh embodiment isdifferent from the server 4 according to the fifth embodiment in that itfurther includes a referring means 46 and a storage means 47.

In the case where the operation mode of the scanning system 1 is“viewing mode”, when the structuring means 42 according to the seventhembodiment receives a set of ID information, ISBN information and pagenumber information from the smartphone 3, it outputs the ISBNinformation and the page number information to the referring means 46.Further, in the case where the operation mode of the scanning system 1is “viewing mode”, when the structuring means 42 receives a set of IDinformation, ISBN information, page number information and bodyinformation from the smartphone 3, it outputs the ISBN information, thepage number information and the body information to the referring means46.

The referring means 46 acquires, based on the information output fromthe structuring means 42, information input for a paper book by otherpeople from the storage means 47 and transmits them to the smartphone 3.

In the storage means 47, information input for a paper book by otherpeople in a social reading service is stored for each of a plurality ofother people. The information is stored into the storage means 47 of theserver 4 when another person inputs the information and transmits it tothe server 4 using another smartphone 3, for example. Note that aninformation processing device that transmits information to be storedinto the storage means 47 to the server 4 is not limited to thesmartphone 3, and it may be a PC or a tablet as described earlier.

A CPU included in the server 4, for example, executes a program storedin the storage means 41 included in the server 4 and thereby operates asthe referring means 46 just like the above-described structuring means42.

To be more specific, in the storage means 47, comment informationpresenting comments or Q&A information presenting Q&A (question &answer) on a whole paper book, comment information presenting commentsor Q&A information presenting Q&A on each page of a paper book, andcomment information presenting comments or Q&A information presentingQ&A on each specified chunk (for example, every specified number ofparagraphs) of a body in each page are stored, for example.

The comment information or the Q&A information on a whole paper book isstored in the storage means 47 in association with ISBN informationindicating the ISBN of the paper book. The comment information or theQ&A information on a certain page in a certain paper book is stored inthe storage means 47 in association with a pair of an ISBN numberindicating the ISBN of the paper book and page number informationindicating the page number of the page. The comment information or theQ&A information on a certain chunk of a body on a certain page in acertain paper book is stored in the storage means 47 in association witha set of an ISBN number indicating the ISBN of the paper book, pagenumber information indicating the page number of the page, and bodyinformation presenting the body.

Thus, as shown in FIG. 30, when ISBN information and page numberinformation are output from the structuring means 42, the referringmeans 46 acquires the comment information or the Q&A information that isassociated with a pair of ISBN information and page number informationthat are the same as the output ISBN information and page numberinformation from the storage means 47 and transmits it to the smartphone3.

Further, as shown in FIG. 30, when ISBN information, page numberinformation and body information are output from the structuring means42, the referring means 46 acquires the comment information or the Q&Ainformation that is associated with a set of ISBN information and pagenumber information that are the same as the output ISBN information andpage number information and body information presenting a bodycontaining the body presented by the body information from the storagemeans 47 and transmits it to the smartphone 3.

As shown in FIG. 29, the smartphone 3 according to the seventhembodiment is different from the smartphone 3 according to the fifthembodiment in that it further includes a display control means 58.

The display control means 58 displays, on the display means 56, an imageshowing comments or Q&A presented by comment information or Q&Ainformation received from the server 4.

A CPU included in the smartphone 3, for example, executes a programstored in the storage means 39 included in the smartphone 3, and therebyoperates as the display control means 58 in the same manner as theabove-described means 30 to 37, 50 to 52 and 54.

On the input means 53 according to the seventh embodiment, a user inputsan instruction to switch the operation mode of the scanning system 1from “reading mode” to “viewing mode”. When an instruction to switch theoperation mode of the scanning system 1 from “reading mode” to “viewingmode” is input to the input means 53 by a user, the information controlmeans 54 switches the operation mode of the scanning system 1 from“reading mode” to “viewing mode”.

In the case where the operation mode of the scanning system 1 is“viewing mode”, when the second selection means 34 transmits IDinformation, ISBN information, page number information, body informationand the like to the server 4, it also transmits notification informationindicating that the operation mode of the scanning system 1 is “viewingmode”. By this notification information, the structuring means 42 of theserver 4 can recognize whether the operation mode of the scanning system1 is “viewing mode” or not.

The effects of the seventh embodiment are described hereinafter withreference to FIG. 31. As shown on the left of FIG. 31, in general socialreading services or online book shopping sites, information such ascomments is presented for each book. On the other hand, in the seventhembodiment, as shown on the right of FIG. 31, information such ascomments in social reading services or online book shopping sites can bepresented for each page of a book or for each body where a page of abook is separated into a specified number of parts. It is thus possibleto present more precise information compared with general social readingservices or online book shopping sites.

Note that, although an example in which the operation mode of thescanning system 1 is switched by an input to the smartphone 3 isdescribed above, it is not limited thereto. The operation mode of thescanning system 1 may be switched by an input of a gesture through thepen scanner 2. For example, the first sub-processing means 50 mayperform processing to switch the operation mode of the scanning system 1between “reading mode” and “viewing mode”.

Further, for example, the second sub-processing means 51 may performprocessing to switch the operation mode of the scanning system 1 between“reading mode” and “second viewing mode”. In “second viewing mode”,another processing means (not shown) of the smartphone 3 may combine aplurality of images presented by a plurality of pieces of imageinformation, perform OCR on a character sequence contained in the imagegenerated by the combining, and generate text information where thecharacter sequence is presented in the form of an electronic text asISBN information indicating ISBN. The second selection means 34transmits the generated ISBN information to the server 4.

The structuring means 42 transmits the ISBN information received fromthe smartphone 3 to the referring means 46. Then, as shown in FIG. 30,when the ISBN information is output from the structuring means 42, thereferring means 46 may acquire comment information or Q&A informationassociated with ISBN information that is the same as the received ISBNinformation from the storage means 47 and transmit it to the smartphone3.

Further, the text information generated by the above-described anotherprocessing means may be generated as reviewer name informationindicating a reviewer name and transmitted to the server 4. In thiscase, comment information or Q&A information are stored in the storagemeans 47 in association with reviewer name information indicating areviewer name who has entered comments or Q&A presented by the commentinformation or the Q&A information. Then, the referring means 46 mayacquire, from the storage means 47, the comment information or the Q&Ainformation that is associated with reviewer name information that isthe same as the reviewer name information received from the smartphone 3and transmit it to the smartphone 3.

Furthermore, the text information generated by the above-describedanother processing means may be generated as chapter title informationindicating a title of a chapter and transmitted to the server 4. In thiscase, the chapter title information is stored in the storage means 47 inassociation with comment information or Q&A information presentingcomments or Q&A on the chapter with the title indicated by the chaptertitle information. Then, the referring means 46 may acquire, from thestorage means 47, the comment information or the Q&A information that isassociated with chapter title information that is the same as thechapter title information received from the smartphone 3 and transmit itto the smartphone 3.

As described above, in the seventh embodiment, for each of a pluralityof paper books, information input by other people on the content of eachpage of the paper book is stored in the storage means 47. The referringmeans 46 acquires information input by other people on the content of apage with a page number indicated by generated page number informationin a paper book corresponding to ISBN specified most recently. Thedisplay control means 58 displays the information acquired by thereferring means 46 on the display means 56.

In this configuration, it is possible to refer to information input byother people on a paper book without a complicated operation. Thus, byusing information in a social reading service as information to bestored in the storage means 47, for example, it is possible to makeactive use of the social reading service.

Eighth Embodiment

An eighth embodiment is described hereinafter. In the followingdescription, the same matter as in the third embodiment described aboveis denoted by the same reference symbol or the like, and the descriptionthereof is omitted as appropriate. The schematic configuration of thescanning system 1 and the hardware configuration of the pen scanner 2according to the eighth embodiment are the same as the schematicconfiguration of the scanning system 1 and the hardware configuration ofthe pen scanner 2 according to the first embodiment and thus notredundantly described.

In the eighth embodiment, as shown in FIG. 32, a time from the end ofscanning of a body to the start of scanning of the next body ismeasured, and when the measured time is within a specified timeinterval, those two bodies are joined and stored. In this configuration,even when one sentence “Aaaaaaaaabbbbbbbbbccc.” is spread over aplurality of lines, it can be stored as one sentence as shown in FIG.32, for example.

Specifically, it is assumed that, in a paper book, “Aaaaaaaaa” iswritten in the first line, “bbbbbbbbb” is written in the second line,and “ccc.” is written in the third line as shown in FIG. 32. In thiscase, in the case where each of a time from when scanning ends at theend point b of the first line to when scanning starts at the start pointc of the second line, and a time from when scanning ends at the endpoint d of the second line to when scanning starts at the start point eof the third line is within a specified time interval, the charactersequences of the respective lines are joined into one sentence andstored.

The detailed configuration of the scanning system 1 according to theeighth embodiment is described hereinafter with reference to FIG. 33.

As shown in FIG. 33, the pen scanner 2 according to the eighthembodiment is different from the pen scanner 2 according to the thirdembodiment in that it further includes a time measurement means 29.

The time measurement means 29 measures a time from the end of scanningof a medium to the start of next scanning of a medium. Stateddifferently, the time measurement means 29 measures a time from pen-upof the pen scanner 2 to pen-down of the pen scanner 2. For example, theMCU 13 that includes a timer (not shown) operates as the timemeasurement means 29. The time measurement means 29 generates timeinformation indicating the measured time and transmits it to thesmartphone 3.

As shown in FIG. 33, the smartphone 3 according to the eighth embodimentis different from the smartphone 3 according to the third embodiment inthat it further includes an input means 53, a time determination means59, and a threshold setting means 60.

When the attribute of information scanned last time is “body” and theattribute of information scanned this time is also “body”, theprocessing method determination means 30 according to the eighthembodiment directs the time determination means 59 to determine a timefrom the end of scanning of a body last time to the start of scanning ofa body this time. In other words, when the processing methoddetermination means 30 determines to process a plurality of pieces ofimage information by the processing method performed by the thirdprocessing means 35 both last time and this time, it gives the timedetermination means 59 an instruction to determine a time from the endof scanning of a body last time to the start of scanning of a body thistime.

The time determination means 59 determines whether the time indicated bythe time information received from the pen scanner 2 is equal to or lessthan a threshold or not. When the time determination means 59 determinesthat the time indicated by the time information is equal to or less thana threshold, it outputs instruction information that gives aninstruction to join the bodies to the second selection means 34.

When the instruction information that gives an instruction to join thebodies is output from the time determination means 59, the secondselection means 34 generates body information presenting a body where abody presented by body information generated this time and a bodypresented by body information generated last time and stored in thestorage means 39 are joined. Then, when the second selection means 34transmits the body information to the server 4, it adds, to the bodyinformation, instruction information that gives an instruction toreplace the body information and then transmits them to the server 4.

When the instruction information is added to the body informationreceived from the smartphone 3, the structuring means 42 according tothe eighth embodiment updates the body information stored last time inthe storage means 41 to the body information received from thesmartphone 3.

The input means 53 is substantially the same as the one described in thefifth to seventh embodiment. Note that, however, the input means 53according to the eighth embodiment outputs the generated input signal tothe threshold setting means 60.

When a user inputs a threshold to the input means 53, the thresholdsetting means 60 acquires the threshold input by the user and sets theacquired threshold to the time determination means 59 based on an inputsignal that is input from the input means 53 in response to the input.Specifically, the threshold to be used by the time determination means59 is updated to the threshold input by a user.

A CPU included in the smartphone 3, for example, executes a programstored in the storage means 39 included in the smartphone 3, and therebyoperates as the time determination means 59 and the threshold settingmeans 60 in the same manner as the above-described means 30 to 36.

A scanning process of the scanning system 1 according to the eighthembodiment is described hereinafter with reference to FIGS. 34 and 35.

The processing steps S11 to S15 are the same as those of the thirdembodiment. However, in the eighth embodiment, the time measurementmeans 29 resets a measurement time between the processing step S14 andthe processing step S15 (S72), and acquires a measurement time betweenthe processing step S12 and the processing step S13 (S71). The timemeasurement means 29 thereby acquires a time from the end of scanninglast time (Yes in S14) to the start of scanning this time (Yes in S12).

The processing steps S16 to S21, S31-S34, S36, S37 and S41 to S48 arethe same as those described in the third embodiment. After the step S34,the time determination means 59 determines whether the time acquired bythe time measurement means 29 is equal to or less than a threshold(S73).

When the time acquired by the time measurement means 29 is not equal toor less than a threshold (No in S73), the processing steps S44 to S46are performed.

On the other hand, when the time acquired by the time measurement means29 is equal to or less than a threshold (Yes in S73), the secondselection means 34 generates body information presenting a body joininga body presented by body information stored in the storage means 39 anda body presented by body information generated by the third processingmeans 35 (S74). The second selection means 34 stores the generated bodyinformation into the storage means 39. In other words, the body that isscanned most recently is updated to the body that is newly scanned(S75). The structuring means 42 updates the body information stored lasttime in the storage means 41 to the body information generated by thesecond selection means 34 (S76).

Note that, although an example where body information are joined when ameasured time is equal to or less than a threshold, and body informationare not joined and stored as separate body information (for example,separate text files) in the storage means 41 when a measured time is notequal to or less than a threshold is described above, it is not limitedthereto. For example, when a measured time is equal to or less than athreshold, bodies presented by two body information may be joined intoone body information (for example, one text file) without inserting anewline character between them, and when a measured time is not equal toor less than a threshold, bodies presented by two body information maybe joined into one body information by inserting a newline characterbetween them.

Note that, although an example where the determination as to whether atime acquired by the time measurement means 29 is equal to or less thana threshold and the setting of a threshold are performed in thesmartphone 3 is described above, it is not limited thereto. For example,the determination as to whether a time acquired by the time measurementmeans 29 is equal to or less than a threshold and the setting of athreshold may be performed in the pen scanner 2. Specifically, the penscanner 2 may include the input means 53, the time determination means59 and the threshold setting means 60. In this case, the pen scanner 2has, as the input means 53, an operation button for inputting athreshold, for example.

Further, although an example where the time measurement means 29, theinput means 53, the time determination means 59 and the thresholdsetting means 60 are applied to the third embodiment is described above,it is not limited thereto. The time measurement means 29, the inputmeans 53, the time determination means 59 and the threshold settingmeans 60 may be applied to any of the fourth to seventh embodiments.

As described above, in the eighth embodiment, the time measurement means29 measures a time from the end of scanning of a medium last time to thestart of scanning of a medium this time. When the body information isgenerated both last time and this time, the time determination means 59determines whether the time measured by the time measurement means 29 isequal to or less than a specified threshold. When it is determined thatthe time measured by the time measurement means 29 is equal to or lessthan a specified threshold, the structuring means 42 updates bodyinformation generated last time to body information presenting a bodyjoining a body presented by the body information generated last time anda body presented by body information generated this time. When, on theother hand, it is determined that the time measured by the timemeasurement means 29 is not equal to or less than a specified threshold,the structuring means 42 stores body information generated this timeinto the storage means 41 without joining it to the body informationgenerated last time.

In this configuration, it is possible to join and store a body that isspread over a plurality of lines without a complicated operation.

The above-described program can be stored and provided to the computerusing any type of non-transitory computer readable medium. Thenon-transitory computer readable medium includes any type of tangiblestorage medium. Examples of the non-transitory computer readable mediuminclude magnetic storage media (such as floppy disks, magnetic tapes,hard disk drives, etc.), optical magnetic storage media (e.g.magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R/W, andsemiconductor memories (such as mask ROM, PROM (Programmable ROM), EPROM(Erasable PROM), flash ROM, RAM (Random Access Memory), etc.). Theprogram may be provided to a computer using any type of transitorycomputer readable medium. Examples of the transitory computer readablemedium include electric signals, optical signals, and electromagneticwaves. The transitory computer readable medium can provide the programto a computer via a wired communication line such as an electric wire oroptical fiber or a wireless communication line.

Although embodiments of the present invention are described specificallyin the foregoing, the present invention is not restricted to theabove-described embodiments, and various changes and modifications maybe made without departing from the scope of the invention.

For example, although the second processing means 33 and the fourthprocessing means 36 combine a plurality of images presented by aplurality of pieces of image information and compress the imagegenerated by the combining, the present invention is not limitedthereto. Image information presenting the image generated by thecombining may be transmitted to the server 4 and stored without beingcompressed.

The first to eighth embodiments described above can be combined asdesirable by one of ordinary skill in the art.

While the invention has been described in terms of several embodiments,those skilled in the art will recognize that the invention can bepracticed with various modifications within the spirit and scope of theappended claims and the invention is not limited to the examplesdescribed above.

Further, the scope of the claims is not limited by the embodimentsdescribed above.

Furthermore, it is noted that, Applicant's intent is to encompassequivalents of all claim elements, even if amended later duringprosecution.

What is claimed is:
 1. A scanning system comprising: a processorconfigured to execute: an image acquisition unit that acquires aplurality of pieces of image information generated by continuouslyscanning a medium to be scanned by a scanning device; a pathdetermination unit that determines a path of the scanning device basedon moving vectors, corresponding to a movement of the scanning deviceacross the medium, generated from the plurality of pieces of imageinformation acquired by the image acquisition unit; a processingtechnique determination unit that determines a processing technique forthe plurality of pieces of image information in accordance with the pathof the scanning device determined by the path determination unit; and aprocessing unit that processes the plurality of pieces of imageinformation according to the processing technique determined by theprocessing technique determination unit to convert the plurality ofpieces of image information into information in a form corresponding tothe processing method.
 2. The scanning system according to claim 1,wherein the processing technique determination unit determines differentprocessing techniques depending on whether the path of the scanningdevice is linear or not.
 3. The scanning system according to claim 2,further comprising: a button to be pressed by a user, wherein theprocessing technique determination unit further determines differentprocessing techniques depending on whether the button is pressed or notduring scanning of the medium.
 4. The scanning system according to claim3, wherein the processing technique determination unit determines aprocessing technique for the plurality of pieces of image informationfrom a plurality of processing techniques prepared in advance, and theplurality of processing techniques include a first processing techniquethat generates information after conversion of the plurality of piecesof image information as page number information presenting a page numberof a paper book scanned as the medium, a second processing techniquethat generates information after conversion of the plurality of piecesof image information as drawing information presenting a drawing of apaper book scanned as the medium, a third processing technique thatgenerates information after conversion of the plurality of pieces ofimage information as body information presenting a body of a paper bookscanned as the medium, and a fourth processing technique that generatesinformation after conversion of the plurality of pieces of imageinformation as note information presenting, as notes, the path of thescanning device determined based on the plurality of pieces of imageinformation.
 5. The scanning system according to claim 4, furthercomprising: a storage unit, wherein the processor is further configuredto execute: a structuring unit that stores, into the storage unit, pagenumber information generated most recently in association with drawinginformation, body information or note information generated aftergeneration of the page number information.
 6. The scanning systemaccording to claim 5, wherein the structuring unit stores, into thestorage unit, page number information generated most recently and bodyinformation or drawing information generated most recently inassociation with note information generated after generation of the pagenumber information and the body information.
 7. The scanning systemaccording to claim 5, further comprising: an input unit that receives aninput to specify ISBN (International Standard Book Number) of a paperbook to be scanned as the medium, wherein the structuring unit stores,into the storage unit, ISBN information indicating ISBN specified mostrecently and page number information generated most recently inassociation with drawing information, body information or noteinformation generated after specifying of the ISBN and generation of thepage number information.
 8. The scanning system according to claim 7,further comprising: a pen scanner that includes the image acquisitionunit, the pen scanner comprising an ID storage unit to store in advanceID information uniquely identifying the pen scanner, and an IDtransmitting unit that transmits ID information stored in the ID storageunit, wherein the structuring unit stores, into the storage unit, IDinformation transmitted from the ID transmitting unit, ISBN informationindicating ISBN specified most recently and page number informationgenerated most recently in association with drawing information, bodyinformation or note information generated after receiving of the IDinformation, specifying of the ISBN and generation of the page numberinformation.
 9. The scanning system according to claim 8, wherein thestorage unit further stores a plurality of electronic books officiallygenerated from a plurality of paper books, and the scanning systemfurther comprises a referring unit that, when a body presented by thegenerated body information partially matches a body on a page with apage number indicated by the page number information generated mostrecently in an electronic book corresponding to the ISBN specified mostrecently, corrects the body presented by the generated body informationso as to exactly match the body on the page with the page numberindicated by the page number information generated most recently. 10.The scanning system according to claim 9, wherein the storage unitfurther stores a plurality of sets of the ISBN information, the pagenumber information and the body information generated when other usersscan the medium, and when an electronic book corresponding to the ISBNspecified most recently is not stored in the storage unit, and a bodypresented by the generated body information partially matches at leastone of bodies presented by body information of a plurality of otherusers associated with ISBN information and page number information beingthe same as the ISBN information indicating the ISBN specified mostrecently and the page number information generated most recently, thereferring unit corrects the body presented by the generated bodyinformation so as to exactly match a body with the largest number ofmatches among the bodies presented by the body information of theplurality of other users.
 11. The scanning system according to claim 7,wherein the storage unit further stores, for each of a plurality ofpaper books, information input by other people on content of each pageof the paper books, and the scanning system further comprises a displaydevice, wherein the processor is further configured to execute: areferring unit that acquires information input by other people oncontent of a page with a page number indicated by the generated pagenumber information in a paper book corresponding to the ISBN specifiedmost recently; and a display control unit that displays informationacquired by the referring unit on the display device.
 12. The scanningsystem according to claim 5, further comprising: a storage unit to storegesture information indicating a gesture being a template of the path ofthe scanning device, wherein the processor is further configured toexecute: a referring unit that performs sub-processing that changes anoperation mode of the scanning system from normal mode to ISBN scan modewhen determining that a path of scanning the medium corresponds to agesture indicated by the gesture information; and a processing meansthat, when the operation mode of the scanning system is ISBN scan mode,converts the plurality of pieces of image information into informationin a specified form, and generates information after conversion as ISBNinformation indicating ISBN of a paper book scanned as the medium,wherein the structuring unit stores, into the storage unit, ISBNinformation generated most recently and page number informationgenerated most recently in association with drawing information, bodyinformation or note information generated after generation of the ISBNinformation and the page number information.
 13. The scanning systemaccording to claim 5, wherein the processor is further configured toexecute: a time measurement unit that measures a time from end ofscanning of the medium last time to start of scanning of the medium thistime; and a time determination unit that determines whether a timemeasured by the time measurement unit is equal to or less than aspecified threshold when body information is generated both last timeand this time, wherein when a time measured by the time measurement unitis determined to be equal to or less than a specified threshold, thestructuring unit updates body information generated last time to bodyinformation presenting a body joining a body presented by the bodyinformation generated last time and a body presented by body informationgenerated this time, and when a time measured by the time measurementunit is determined to be not equal to or less than a specifiedthreshold, the structuring unit stores body information generated thistime into the storage unit without joining it to body informationgenerated last time.
 14. The scanning system according to claim 1,wherein the processing technique determination unit determines aprocessing technique for the plurality of pieces of image informationfrom a plurality of processing techniques prepared in advance, and theplurality of processing techniques include a first processing techniquethat performs optical character recognition on an image combining aplurality of images presented by the plurality of pieces of imageinformation to convert the plurality of pieces of image information intotext information, and a second processing technique that converts theplurality of pieces of image information into image informationpresenting an image combining a plurality of images presented by theplurality of pieces of image information.
 15. The scanning systemaccording to claim 14, further comprising: a storage unit to storegesture information indicating a gesture being a template of the path ofthe scanning device, wherein the processor is further configured toexecute: a referring unit that performs specified sub-processing whendetermining that a path of scanning the medium corresponds to a gestureindicated by the gesture information.
 16. The scanning system accordingto claim 14, further comprising: a display device, wherein the processoris further configured to execute: a display control unit that displays,on the display device, a notification of a processing techniquedetermined by the processing technique determination unit.
 17. A scannedimage processing device comprising: a processor configured to execute: areceiving unit that receives, from a pen scanner, a plurality of piecesof image information generated by continuously scanning a medium to bescanned by the pen scanner; a processing technique determination unitthat determines a processing technique for the plurality of pieces ofimage information in accordance with a path of the pen scannerdetermined based on moving vectors, corresponding to a movement of thepen scanner across the medium, generated from the plurality of pieces ofimage information received by the receiving unit; and a processing unitthat processes the plurality of pieces of image information by aprocessing technique determined by the processing techniquedetermination unit to convert the plurality of pieces of imageinformation into information in a form corresponding to the processingtechnique.
 18. A scanning method comprising: acquiring a plurality ofpieces of image information generated by continuously scanning a mediumto be scanned by a scanning device; determining a path of the scanningdevice based on moving vectors, corresponding to a movement of thescanning device across the medium, generated from the plurality ofpieces of image information; determining a processing technique for theplurality of pieces of image information in accordance with the path ofthe scanned device; and processing the plurality of pieces of imageinformation by the determined processing technique to convert theplurality of pieces of image information into information in a formcorresponding to the processing technique.